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Preface 



"Artificial Intelligence" is an area of computer science that aims to develop important 
concepts for simulation of human intelligence whereas "Applied Intelligence" focuses 
on issues involving solutions of real-life problems, arising in defense, government, 
management, manufacturing and many other areas which are too complex to solve 
through conventional approaches. The objective of the International Conference on 
Industrial, Engineering & Other Applications of Applied Intelligent Systems 
(IEA/AIE) is to promote and disseminate recent research developments in Applied 
Intelligence. It accepts quality papers on a wide range of topics in all disciplines 
including science, social science, art, engineering, and finance. Held yearly, the 
conference brings together scientists, engineers and practitioners, who work on 
designing and developing applications that use intelligent techniques and apply them 
to variety of application domains. 

This book is published in the "Studies in Computational Intelligence" series by 
Springer Verlag. It is comprised of eighteen chapters providing up-to-date and state- 
of-the-art research on the applications of Applied Intelligence techniques. Each 
chapter contains a short paper authored by a participant of the 24th International 
Conference on Industrial, Engineering & Other Applications of Applied Intelligent 
Systems (1EA/A1E-2011). Each chapter was reviewed by at least two anonymous 
referees to assure the high quality. 

The book covers a wide range of topics including Adaptive Learning, Belief 
Management, Multi-agent Systems, Genetic Algorithms, Neural Networks, and Fuzzy 
Modeling. This book provides useful reference values and readers will find 
opportunities as well as recognize challenges in the development and applications of 
intelligent systems. 

We would like to thank Springer Verlag and Professor Janusz Kacprzyk, the editor 
of this series, for their help in publishing the book. We would also like to thank the 
reviewers for their hard work in assuring the high quality of the book. Finally, we 
cordially thank all the authors, who did important contributions to the book. Without 
their efforts, this book cannot be successfully carried out. 



April 20 1 1 Kishan G. Mehrotra 

Chilukuri Mohan 

Jae C. Oh 

Pramod K. Varshney 

Moonis Ali 



Contents 



Section 1: Clustering, Data Refinement, and 
Applications 

An Asymmetric Criterion for Cluster Validation 1 

Hosein Alizadeh, Behrouz Minaei, Hamid Parvin, Mohsen Moshki 

Discovery of Domain Values for Data Quality Assurance 15 

Lukasz Ciszak 

Dynamic Time Warping-Based K-Means Clustering for 
Accelerometer-Based Handwriting Recognition 21 

Minsu Jang, Mun-Sung Han, Jae-hong Kim, Hyun-Seung Yang 

On Segmentation of Interaction Values 27 

Nguyen Chi Lam, Hiep Xuan Huynh, Fabrice Guillet 

SI-APRENDE: An Intelligent Learning System Based on 

SCORM Learning Objects for Training Power Systems 

Operators 33 

Liliana Argotte, G. Arroyo-Figueroa, Julieta Noguez 



Section 2: Neural Network, Support Vector 
Machines, and Genetic Algorithm 

Least Squares Support Vector Machines for Channel 

Prediction in the MIMO System 39 

Jerzy Martyna 



VIII Contents 

A Neural Computation Model Based on nCRF and Reverse 
Control Mechanism 45 

Hui Wei, Xiao-Mei Wang 

Particle Swarm Optimization of a Recurrent Neural Network 

Control for an Underactuated Rotary Crane with Particle 

Filter Based State Estimation 51 

Sam Chau Duong, Hiroshi Kinjo, Eiho Uezato, Tetsuhiko Yamamoto 

Extracting Connection Types in Process Models Discovered 

by Using From-to Chart Based Approach 59 

Eren Esgin, Pinar Senkul 

Using Class Decomposition for Building GA with Fuzzy 
Rule-Based Classifiers 67 

Passent El-Kafrawy, Amr Sauber 



Section 3: Game Theory, Optimization, Scheduling, 
and Routing 

Improving the Performance of Multi-start Search on the 

Traveling Salesman Problem 77 

Charles R. King, Mark McKenney, Dan E. Tamir 

A Constraint Programming Application for Rotating 

Workforce Scheduling 83 

Markus Triska, Nysret Musliu 

An Agent-Based Architecture for Multifaceted Online Dispute 
ResolutionTools 89 

Davide Carneiro, Paulo Novais, Jose Neves 

Towards an Agent-Based Knowledge Management System 95 

Marwan Altawil, Asma Moubaiddin, Nadim Obeid 

An Approach to Auction-Based Web Server Admission 

Control 101 

Jolanta Wrzuszczak-Noga, Leszek Borzemski 



Section 4: Robotics and Navigation 

Adaptive Fuzzy Rules Emulated Networks Controller for 

7-DOF Robotic Arm with Time Varying Learning Rate 107 

C. Treesatayapun 



Contents IX 

Belief Management for Autonomous Robots Using 

History-Based Diagnosis 113 

Stephan Gspandl, Ingo Pill, Michael Reip, Gerald Steinbauer 

Enhancing Believability of Virtual Soccer Players: Application 

of a BDI-Model with Emotions and Trust 119 

Tibor Bosse, Daniel Hohle 

Author Index 129 



An Asymmetric Criterion for Cluster Validation 



Hosein Alizadeh, Behrouz Minaei, Hamid Parvin, Mohsen Moshki 

School of Computer Engineering, Iran University of Science and Technology (IUST), 

Tehran, Iran 

{halizadeh,b_minaei , parvin, moshki }@iust . ac . ir 



Abstract. Many stability measures to validate a cluster have been proposed 
such as Normalized Mutual Information. The drawback of the common 
approach is discussed in this paper and then a new asymmetric criterion is 
proposed to assess the association between a cluster and a partition which is 
called Alizadeh-Parvin-Moshki-Minaei criterion, APMM. The APMM criterion 
compensates the drawback of the common Normalized Mutual Information 
(NMI) measure. Also, a clustering ensemble method is proposed which is based 
on aggregating a subset of primary clusters. This method uses the Average 
APMM as fitness measure to select a number of clusters. The clusters which 
satisfy a predefined threshold of the mentioned measure are selected to 
participate in the clustering ensemble. To combine the chosen clusters, a co- 
association based consensus function is employed. Since the Evidence 
Accumulation Clustering, EAC, method cannot derive the co-association matrix 
from a subset of clusters, a new EAC based method which is called Extended 
EAC, EEAC, is employed to construct the co-association matrix from the 
chosen subset of clusters. The empirical studies show that the proposed method 
outperforms other ones. 

Keywords: Clustering Ensemble, Stability Measure, Improved Stability, 
Evidence Accumulation, Extended EAC, Co-association Matrix, Cluster 
Evaluation. 



1 Introduction 

Data clustering or unsupervised learning is an important and very challenging 
problem. The objective of clustering is to partition a set of unlabeled objects into 
homogeneous groups or clusters [10]. There are many applications which use 
clustering techniques to discover structures in data, such as data mining [10], 
information retrieval, image segmentation, and machine learning [12]. In real- world 
problems, the clusters can appear with different shapes, sizes, data sparseness, and 
degrees of separation. Clustering techniques require the definition of a similarity 
measure between patterns. On account of lack of any prior knowledge about cluster 
shapes, choosing a specialized clustering method is not easy [24]. Studies in the last 
few years have tended to combinational methods. Cluster ensemble methods attempt 
to find a better and more robust clustering solution by fusing information from several 
primary data partitionings [1]. 

K.G. Mehrotra et al. (Eds.): Developing Concepts in Applied Intelligence, SCI 363, pp. 1^14| 
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2 H. Alizadeh et al. 

Generally, there are two main steps in clustering ensemble: (a) the creation of some 
weak partitionings, (b) the aggregation of the obtained primary partitioning. The first 
step is the creation of some weak partitionings. Because every primary partitioning 
reveals a hidden aspect of a data, their ensemble can cover their individual drawbacks. 
So, the primary results are needed to be as diverse as possible to give more 
information about the underlying patterns in the data. Many methods have been 
suggested to create the necessary diversity for the primary results. To do this, using 
different clustering algorithms is the simplest way. Some other methods are choosing 
different initialization, different algorithm parameters, subset of features, mapping the 
data to other feature spaces [1], resampling of the data [20]. In this paper the 
resampling, different base algorithms, different initialization and different parameters 
are used to provide the necessary diversity for the primary results. 

The second main step in clustering ensemble is to combine the primary 
partitionings obtained in the first step. The co-association matrix based aggregator is 
one of the most common methods to combine the primary partitionings which is 
employed in this paper too. EAC which is first proposed by Fred and Jain maps the 
individual data partitions in a clustering ensemble into a new between-patterns 
similarity measure, summarizing inter-pattern structures perceived from these 
clusterings. The final data partition is obtained by applying the single-link method to 
this new similarity matrix [13]. 

In this paper a new clustering ensemble method is proposed which uses a subset of 
primary clusters. A new validity measure which is called Improved Stability, 
IStability, is suggested to evaluate the cluster goodness. Each cluster that satisfies a 
threshold of IStability can be considered to participate in constructing the co- 
association matrix. A new method named Extended Evidence Accumulation 
Clustering, EEAC, is proposed to construct this matrix. Finally, a hierarchical method 
is applied over the obtained matrix to extract the final partition. 

2 Background 

The clustering ensemble which is based on a subset of selected primary clusters or 
partitions has a main problem which is the manner of evaluating clusters or partitions. 
As the data clustering is an unsupervised problem, its validation process is the most 
troublesome task. Baumgartner et al. in [2] have presented a resampling based 
technique to validate the results of exploratory fuzzy clustering analysis. Since the 
concept of cluster stability is introduced as a means to assess the validity of data 
partitionings, it has been incrementally used in the literature [14]. This idea which is 
based on resampling method is initially described in [5] and later generalized in 
different ways in [16]. Roth et al. in [24] have proposed a resampling based technique 
to validate a cluster. The basic element in their method which is a complementary 
version of the past methods is cluster stability. The stability measures the association 
between obtained partitions from two individual clustering algorithms. The great 
values of the stability measure mean that applying the clustering algorithm several 
times on a data set probably yields the fixed results [22]. Roth and Lange [23] have 
presented a new algorithm for data clustering which is based on feature selection. In 
their method the resampling based stability measure is used to set the algorithm 
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parameters. There are several cluster validation methods which are based on stability 
concept [19]. Ben-Hur et al. [3] have proposed a technique to exploit the stability 
measurements of the clustering solutions obtained by perturbing a data set. In their 
approach, the stability is characterized by the distribution of the pairwise similarities 
between clusterings obtained from sub samples of the data. First, the co-association 
matrix is acquired using the resampling method. Then, Jaccard coefficient is extracted 
from this matrix as the stability measure. Also, Estivill-Castro and Yang in [9] have 
offered a method by which Support Vector Machines are used to evaluate the 
separation of the clustering results. By filtering noise and outliers, this method can 
identify the robust and potentially meaningful clustering result. 

Moller and Radke [21] have introduced an approach to validate a clustering results 
based on partition stability. This method uses a perturbation which is produced by 
adding some noise to the data. An empirical study robustly indicates that the 
perturbation usually outperforms bootstrapping and subsampling. Whereas the 
empirical choice of the subsampling size is often difficult [7], the choosing of the 
perturbation strength is not so crucial. This method uses a Nearest Neighbor 
Resampling approach (NNR) that offers a solution to both problems of information 
loss and empirical control of the change degree made to the original data. The NNR 
techniques were first used for time series analysis [4]. Inokuchi et al. [17] have 
proposed a kernelized validity measures where a kernel means the kernel function 
used in support vector machines. Two measures are considered in this measure. One 
is the sum of the traces of the fuzzy covariances within clusters and the second is a 
kernelized Xie-Beni's measure [26]. This validity measure is applied to the 
determination of the number of clusters and also the evaluation of robustness of 
different partitionings. Das and Sil [6] have proposed a method to determine the 
number of clusters which validates the clusters using splitting and merging technique 
in order to obtain optimal set of clusters. 

Fern and Lin [11] have suggested a clustering ensemble approach which selects a 
subset of solutions to form a smaller but better performing cluster ensemble than 
using all primary solutions. The ensemble selection method is designed based on 
quality and diversity, the two factors that have been shown to influence the cluster 
ensemble performance. This method attempts to select a subset of primary partitions 
that simultaneously has both the highest quality and diversity. The Sum of 
Normalized Mutual Information, SNMI [25], [12] and [13], is used to measure the 
quality of individual partition with respect to other ones. Also, the Normalized Mutual 
Information, NMI, is employed to measure the diversity between partitions. Although 
the ensemble size in their method is relatively small, this method can achieve a 
significant performance improvement over full ensembles. Law et al. propose a multi 
objective data clustering method based on the selection of individual clusters 
produced by several clustering algorithms, through an optimization procedure [18]. 
This technique chooses the best set of objective functions for different parts of the 
feature space from the results of base clustering algorithms. Fred and Jain [14] have 
offered a new clustering ensemble method that learns the pairwise similarity between 
points in order to facilitate a proper partitioning of the data without the a priori 
knowledge of the number of clusters and of the shape of the clusters. This method 
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which is based on cluster stability evaluates the primary clustering results instead of 
final clustering. 



3 Proposed Clustering Ensemble 

In this section, first the proposed clustering ensemble method is briefly outlined, and 
then its phases are described in the subsequent subsections in more detail. 
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Fig. 1. The proposed clustering ensemble method. 



The main idea of the proposed clustering ensemble method is to utilize a subset of 
the best performing primary clusters in the ensemble instead of all of them. It seems 
that every cluster does not have a good quality. So, in this method just those clusters 
which satisfy enough stability to participate in the combination are chosen. The 
cluster selection is done based on cluster stability which is defined according to 
Normalized Mutual Information, NMI. 

Fig. 1 depicts the proposed clustering ensemble procedure. First, a set of B primary 
partitions is provided using ^-means and Linkage methods to create the necessary 
diversity for an ensemble. Then, the Improved Stability, IStability, is computed for all 
clusters of each obtained partitions. The manner of computing IStability is described 
in next sections in detail. After that, a subset of the most stable clusters is selected to 
participate in the final decision committee. This is done simply by applying a 
threshold over the IStability value of any cluster. In the next step, the selected clusters 
construct the co-association matrix. Several methods have been proposed how to 
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combine the primary results [25], [13] and [15]. Here, the difference is the absence of 
some clusters in primary partitions. Since the original EAC method [13] cannot truly 
identify the pairwise similarity while there is just a subset of clusters, in this paper a 
new method to construct the co-association matrix is presented which hereafter called: 
Extended Evidence Accumulation Clustering method, EEAC. Finally, the single-link 
method is employed to extract the final clusters from this matrix. 

3.1 Cluster Evaluation Using APMM Criteria 

A stable cluster is the one that has a high likelihood of recurrence across multiple 
applications of the clustering method. The stable clusters are usually preferable, since 
they are robust with respect to minor changes in the dataset [18]. 

Now assume that we want to compute the stability of cluster C,. In this method first 
a set of partitionings over resampled datasets is provided which is called the reference 
set. In this notation D is resampled data and P(D) is a partitioning over D. Now, the 
problem is: "How many times is the cluster C, repeated in the reference partitions?" 
Denote by NMI(C,-,P(Z))), the Normalized Mutual Information between the cluster C, 
and a reference partition P(D). The most previous works only compare a partition 
with another partition [25]. However, the stability used in [18] evaluates the 
similarity between a cluster and a partition by transforming the cluster C, to a 
partition and employing common partition to partition methods. To illustrate this 
method let P\ = P d ={Ci,D/CJ be a partition with two clusters, where D/C, denotes the 
set of data points in D that are not in C,. 

Then we may compute a second partition P 2 =P h ={C*,D/C*J, where C* denotes the 
union of all "positive" clusters in P(D) and others are in D/C . A cluster C, in P(D) is 
positive if more than half of its data points are in C,. 

Now, define NMI(C,,P(£>)) by NMI(P fl ,P i ) which is calculated as [12]: 
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where n is the total number of samples and y denotes the number of shared patterns 

between clusters deP" and C'jeP". 

This is done between the cluster C, and all partitions available in the reference set. 
Fig. 2 shows this method. 

NMIj in Fig. 2 shows the stability of cluster C, with respect to the j'-th partition in 
reference set. The total stability of cluster C, is defined as: 

1 M 
Stability (C t ) = — ^ NMI t (2) 

,=i 

where M is the number of partitions available in reference set. This procedure is 
applied to each cluster of every primary partition. 
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Fig. 2. Computing the Stability of Cluster C, 
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Fig. 3. Two primary partitions with k=3. (a) True clustering, (b) Spurious clustering. 



This approach to evaluate the stability has some drawbacks which can appear 
under some circumstances. Here, a drawback of this procedure is introduced. Then an 
approach which is based on a novel asymmetric measure named Alizadeh-Parvin- 
Moshki-Minaei, APMM criteria, is proposed. 

Fig. 3 shows two primary partitions with the evaluated stability of each cluster and 
the process of its computation. In this example K-means is applied as the base 
clustering algorithm and the true number of clusters, k=3, is forced. For this example 
the number of all partitions in reference set is 40. In 36 partitions the result is 
relatively similar to Fig 3. a and in 4 times the top left cluster is divided into two 
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clusters, like Fig 3.b. Fig. 3a shows a true clustering. Since the well separated cluster 
in the top left corner is repeated several times (90% repetition) in partitionings of the 
reference set, it has to acquire a great stability value (but not equal to 1), however it 
acquires the stability value of 1. On account of the two clusters in right hand of 
Fig. 3a are relatively jointed and sometimes they are not recognized in reference set as 
well, they acquire less stability value. Fig. 3.b shows a spurious clustering which the 
two right clusters are incorrectly merged. Since a fixed number of clusters are forced 
to the base algorithm, consequently the top left cluster is divided into two clusters. 
Here the drawback of the stability measure is significantly appeared. Although it is 
obvious that this partition and correspondingly the right big cluster is rarely appeared 
in reference set (10% repetition), the stability of the right big cluster is evaluated 
equal to 1 . Since the NMI is a symmetric equation, the stability of the top left cluster 
in Fig 3. a is exactly equal to the big right cluster in Fig 3.b; however they are repeated 
90% and 10% respectively. In other word, when two clusters are complement of each 
other, their stabilities are always equal. This drawback occurs when the number of 
positive clusters in the considered partition of reference set is greater than 1. It means 
when the cluster C* is obtained by merging two or more clusters, the unwelcome 
results in stability value is occurred. 



Clusterl Cluster2 Cluster3 Cluster4 



«.«.•••.• •.?.:. ^<0 





Fig. 4. Evaluating the APMM criterion for cluster C; from clustering (a) with respect to 
clustering (b), with k=4. 



Here, a new criterion is proposed which can solve this problem. Assume that the 
problem is evaluating the APMM criterion for cluster C] in Fig. 4a with respect to 
clustering obtained in Fig. 4b. 

The main idea in this method is to eliminate the symmetricalness which exists in 
NMI equation. In this approach, except the cluster C] all other clusters in P" are taken 
out. Also, all clusters in P* which are not included the samples of this cluster are 
eliminated. In the next step, the other samples which are not in Cy of P", are removed 
from clusters in P 1, (from the clusters which include some of these samples). This 
process is depicted in Fig. 5. 
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Fig. 5. Providing data for evaluating the APMM criterion, (a) Deleting all other clusters except 
C; from P". (b) deriving P , the corresponding samples of C; in P . 



Now, the entropy between remained clusters in two partitions P" and P b is 
computed (see Fig. 6). On account of the other involved samples are eliminated, this 
criterion is not symmetric. 

All the previous works are based on the NMI definition as equation 1. Even for 
evaluating the occurrence of a cluster in a partition, the problem is modified in some 
way to become the comparing problem between two partitions and then the NMI 
equation is used. In this paper, the problem is not changed according to definition of 
NMI; instead, the NMI equation is modified so that the occurrence of a cluster in a 
partition is computed. It is done by evaluating the entropy between the considered 
cluster and other pseudo clusters in the corresponding partition. In this paper the 
Alizadeh-Parvin-Moshki-Minaei criterion, APMM, is defined between a cluster C, 
from P° and the partition P b from P , as below equation: 



f \ 



2 log 



APMM(C-,P b *) = - 



K n i Ji=^ 



1< 



tflog 



r n a\ K. 



+ 



Z<log 



V " J 7=1 



f b*\ 
v n , 



(3) 
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ab* 



where n is number of samples available in the cluster C, and «, denotes the number 

of shared samples between the clusters Cf e P a and C' e P . Also kb* is the number 
of clusters in P 1 '*. 




Fig. 6. Computing the entropy between the cluster Ci from P" and P from P . 

Here, the Average APMM, AAPMM is proposed as a measure of stability of a 
primary cluster C, with respect to the partitions available in the reference set as 
equation 4: 

1 M 

AAPMM (C, ) = — Yj apmm ( C i . p f ) (4) 

M j=l 

where P. is fromy'-th partition of the reference set. 

3.2 Extended EAC (EEAC) 

In this step, the selected clusters have to construct the co-association matrix. In EAC 
method the m primary results from resampled data are accumulated in an nxn co- 
association matrix. Each entry in this matrix is computed from equation 5. 



C(i,j)- 



m U 



(5) 



Where ny counts the number of clusters shared by objects with indices i andj in the 
partitions over the B clusterings. Also m,j is the number of partitions where this pair of 
objects is simultaneously present. There are only a fraction of all primary clusters 
available, after Thresholding. So, the common EAC method cannot truly recognize 
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the pairwise similarity for erecting the co-association matrix. Each entry of the co- 
association matrix according to definition of the novel method which is called 
Extended Evidence Accumulation Clustering method, EEAC is as equation 6. 



C(i,j) 



i-j 



max(n,- , nj) 



(6) 



Where w, and rij are the number of presence in selected clusters for the ;-th and y'-th 
objects, respectively. Also, n« counts the number of selected clusters which are shared 
by objects with indices i and/ 

To more explain consider the following example. Assume that 5 samples according 
to Fig. 7a, which 4 primary clusterings P 1 to P 4 are applied on this data (according to 
Fig. 7b). 
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Fig. 7. Computing the co-association matrix by EEAC method. (A) Data samples. (B) 4 
primary clusterings. (C) Reminded clusters after applying threshold, th=0.&. 



Also, consider that that stability of the clusters of Fig. 7b is as below: 

Stability (c 2 ) = Stability (c 2 ) = 1 
Stability(c l ) = Stability(c A ) = 1 
Stability (cl) = Stability (c^) = 0.82 
Stability (c{) = Stability (cj 3 ) = 0.55 

By choosing f/i=0.8 the first clusters from P t and P 3 are deleted (Fig. 7c). 

According to equation 6, each entry of co-association matrix is 

2 2 

C(l,2)= = - 

max(2,2) 2 



1 



C(l,3) = C(2,3) : 







C(3,4) = C(3,5) 
C(4,5) 
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max(2,2) 2 

2 2 



max(2,4) 4 
4 4 ^ 



0.5 



max(4,4) 4 
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Before and after applying threshold, the co-association matrix is equal to 
equation 7 and 8, respectively 



C. 



before 



1 


1 


0.5 








1 


1 


0.5 








0.5 


0.5 


1 


0.5 


0.5 








0.5 


1 


1 








0.5 


1 


1 



(7) 



In this matrix the 3 1 object can be considered as both clusters with an equal 
probability 50%. The stability measure adds some info to this matrix by applying the 
threshold. 



C. 



after 



1 


1 











1 


1 

















1 


0.5 


0.5 








0.5 


1 


1 








0.5 


1 


1 



(7) 



By comparing these two matrices and also considering the stability values, it can be 
apprehended that deletion of unstable clusters improves the co-association matrix. In 
other hand, eliminating the unstable cluster with samples {1,2,3} which is spuriously 
created by primary clusterings, enlightens the matrix. 

After erecting the co-association matrix by EEAC method, a consensus function is 
employed to extract the final clusters from the matrix. Here, the single-link method is 
used for this task. 

4 Experimental Results 

This section reports and discusses the empirical studies. The proposed method is 
examined over 5 different standard datasets. It is tried for datasets to be diverse in 
their number of true classes, features and samples. A large variety in used datasets can 
more validate the obtained results. Brief information about the used datasets is 
available in Table 1 . 



Table 1. Brief information about the used datasets. 





Class 


Features 


Samples 


Glass 


6 


9 


214 


Breast-C 


2 


9 


683 


Wine 


3 


13 


178 


Bupa 


2 


6 


345 


Yeast 


10 


8 


1484 
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All experiments are done over the normalized features. It means each feature is 
normalized with mean of and variance of 1, N(0, 1). All of them are reported over 
means of 10 independent runs of algorithm. The final performance of the clustering 
algorithms is evaluated by re-labeling between obtained clusters and the ground truth 
labels and then counting the percentage of the true classified samples. Table 2 shows 
the performance of the proposed method comparing with most common base and 
ensemble methods. 

Table 2. Experimental results. 





Simple Methods (%) 


Ensemble Methods (%) 


Dal a set 


Single 
Linkage 


Average 
Linkage 


Complete 
Linkage 


Kmeans 


Kmeans 
Ensemble 


Full 
Ensemble 


Cluster 
Selection by 
NMI Method 


Cluster 
Selection by 
max Method 


Wine 


37.64 


38.76 


83.71 


96.63 


96.63 


97.08 


97.75 


98.31 


Breast-C 


65.15 


70.13 


94.73 


95.37 


95.46 


95.10 


95.75 


98.33 


Yeasl 


34.38 


35.11 


38.91 


40.20 


45.46 


47.17 


47.17 


47.17 


Glass 


36.45 


37.85 


40.65 


45.28 


47.01 


47.83 


48.13 


50.47 


Bupa 


57.68 


57.10 


55.94 


54.64 


54.49 


55.83 


58.09 


58.40 



The four first columns of Tab. 2 are the results of some base clustering algorithms. 
The results show that although each of these algorithms can obtain a good result over 
a specific data set, it does not perform well over other data sets. For example, 
according to Tab. 2 the K-means algorithm has a good clustering result over Wine 
data set in comparison with linkage methods. But, it has lower performance in 
comparison to linkage methods in the case of Bupa data set. Also, the complete 
linkage has a good performance in Breast-Cancer data set in comparison with others; 
however it is not in the case of all data sets. The four last columns show the 
performance of some ensemble methods in comparison with the proposed one. Taking 
a glance on the last four columns in comparison with the first four columns shows that 
the ensemble methods do better than simple base algorithms in the case of 
performance and robustness against different data sets. The first column of the 
ensemble methods is the results of an ensemble of 100 K-means which is fused by 
EAC method. The 90% sampling from data set is used for creating diversity in 
primary results. The sub-sampling (without replacement) is used as the sampling 
method. Also the random initialization of the seed points of A'-means algorithm helps 
them to be more diverse. The single linkage algorithm is applied as consensus 
function for deriving the final clusters from co-association matrix. The second column 
from ensemble methods is the full ensemble which uses several clustering algorithms 
for generating the primary results. Here, 70 ^f-means with the above mentioned 
parameters in addition to 30 linkage methods provide the primary results. Since 
different runs of a specific linkage method always yield to the same result, there is a 
limitation for using them as the base clustering algorithms. Here, forcing a set of 
different number of clusters, K±2, is used to create diversity, which K is the true 
number of clusters. The used linkage algorithms and their distance criterion are shown 
in Tab. 3. The detailed information about each of these linkage algorithms can be 
found in [8]. 
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The two last columns are the results of ensemble methods which uses only a subset 
of primary clusters. The first one shows the results of an ensemble which uses the 
traditional NMI based stability for cluster validation. The second one is the proposed 
clustering ensemble method which uses the APMM criterion for cluster validation. 
Accumulating the selected clusters in co-association matrix in both methods is done 
using the proposed EEAC method. Finally, the single linkage algorithm is applied 
over co-association matrix to extract final clusters. The primary clustering results are 
provided similar to the full ensemble. In these methods, the threshold which is used to 
cluster selection is determined adaptively. In other words, first it is adjusted to 95%. If 
less than 10% of samples are absent in the selected clusters, the threshold is reduced 
to 90%. This procedure (reducing 5% from threshold value) continues until the 
selected clusters include more than 90% of samples. The results of the two last 
columns show that although these approaches use a subset of the primary clusters, 
they usually outperform the full ensemble. Also, comparing the last two columns 
shows the power of APMM based stability in comparison with the NMI based 
stability. Examinations by 10 independent run over different data sets robustly show 
the quality of the APMM criterion with respect to NMI. 

5 Conclusions 

In this paper a new clustering ensemble method is proposed which is based on a 
subset of total primary spurious clusters. Since the quality of the primary clusters are 
not equal and presence of some of them can even yield to lower performance, here a 
method to select a subset of more effective clusters is proposed. A common cluster 
validity criterion which is needed to derive this subset is based on normalized mutual 
information. In this paper some drawbacks of this criterion is discussed and an 
alternative criterion is suggested which is named Alizadeh-Parvin-Moshki-Minaei, 
APMM. The experiments show that the APMM criterion does slightly better than 
NMI criterion generally; however it significantly outperforms the NMI criterion in the 
case of synthetic data sets. Because of the symmetry which is concealed in NMI 
criterion and also in NMI based stability, it yields to lower performance whenever 
symmetry is also appeared in the data set. Another innovation of this paper is a 
method for constructing the co-association matrix where some of clusters and 
respectively some of samples do not exist in partitions. This new method is called 
Extended Evidence Accumulation Clustering, EEAC. The empirical studies over 
several data sets robustly show that the quality of the proposed method is usually 
better than other ones. 
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Abstract. Data profiling is a crucial step in the data quality process as it 
provides the current data quality rules. In this paper we present experimen- 
tal results comparing our DOMAIN method for the discovery of domain 
constraint values to the commercially available Oracle Warehouse Builder 
(OWB). The experimental results prove that the effectiveness of our ap- 
proach in the discovery of domain values for textual data affected by data 
quality problems is greater than that offered by the OWB. 

Keywords: information quality, data quality, data profiling, domain 
discovery. 



1 Introduction 

Business intelligence systems whose core is a data warehouse require high quality 
data as the input for analytical processes used to draw conclusions regarding 
the future of a given enterprise and support the decision-making process. In this 
paper we concentrate on the experimental comparison of the effectiveness of our 
DOMAIN method for the discovery of domain constraint values (data quality 
rules) to the Oracle Warehouse Builder profiler offering similar functionality. 
The details of DOMAIN are covered by [3]. 

Data quality in an information system is defined as 'the fitness of the data for 
use' [TU]; ontologically, high quality data represent the Universe of Discourse 
(UoD) in the way it allows us to draw conclusions about the future states 
of the UoD [TT] . Real- world data are always affected by various data quality prob- 
lems resulting fronr the imperfection of data input methods (misspelling, mistyp- 
ings), data transport (encoding problems), and finally the evolution of the data 
source (stale and inaccurate metadata). Domain constraints assure that only 
the states that represent valid states of the UoD are allowed in the information 
system. Data quality assessment process discovers the data quality rules and 
assesses the current quality of data in the system. The results are subsequently 
consumed by the data cleansing process that improves the quality of data. 

The data quality improvement is researched in the domain of constraint re- 
pair and consistent query answering [1I2J and record linkage and duplicate elim- 
ination [13] . The profiling methods are studied in terms of the discovery of 

K.G. Mchrotra ct al. (Eds.): Developing Concepts in Applied Intelligence, SCI 363, pp. 151-20. 
springcrlink.com © Springer- Verlag Berlin Heidelberg 2011 
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functional, approximate and conditional dependencies |5I6I7| . and by data ware- 
housing literature |8I9I10| . Commercial tools for the support of profiling activities 
(e.g. Informatica Data Quality, Informatica Data Explorer) do not offer domain 
discovery capabilities; in the ETL tool area, domain discovery options are avail- 
able in the Oracle Warehouse Builder (OWB). To our knowledge, apart from 
ours [3] there has been no research specifically concentrated on the discovery 
of domain constraints. 

2 DOMAIN Method 

In [3] we presented a DOMAIN (DOmain Mining and repAIr in uNclean data) 
method for the discovery of domain values from textual data sets heavily affected 
by various data quality issues. In the remainder of this section we present briefly 
the concepts and the pseudocode of DOMAIN. For the details the reader should 
refer to [3]. 

2.1 Fundamental Concepts 

Let f be a relation in scheme R = {A\, A2, . . . , A n } representing certain collec- 
tion of UoD objects in the way that there are no objects without information 
system representation and each legit state of each object is represented unam- 
biguously by a value from a domain for attribute from R. We will refer to f as 
the ideal relation having the highest possible ontological quality of data. Let r 
be another relation in R such that WL G R dom(Ai) D Dj. 

We assume that the set of values for each attribute in r is defined as 
Di = Di U Ei U Ni, where Ei denotes the set of values that are the 'dam- 
aged' version of correct domain values introduced by the imperfection of entry 
methods or other factors, and Ni is the set of noise, that is, meaningless and 
random values. In the remainder of this paper we will focus on a single attribute 
A and skip the subscript i. We will be considering a multiset Sd = {D,m) where 
m : D — > N is a multiplicity function such that m(d) = \{t G r : t(A) = d}\. 

Let ~~> be a similarity relation on Sd such that 

dk ~> di iff (simki >£ A ration > a), where sim^i is the Jaro- Winkler 
string similarity measure [12] equal for two completely different strings and 
equal 1 for two identical strings, e is a textual similarity threshold, ratioki = 
max ( m(d) ' m(d ) ) ^ s ^ ne "multiplicity ratio, and a is multiplicity ratio thresh- 
old. Informally, element dk is similar to element di in terms of the ~> relation 
if they are textually similar and element is di is significantly more frequent in 
the relation r than dk- The set D and its subsets have the following features: 

Ve G E 3d e D : e ~> d and fid € D : Vn e N n ~-> d. 
The multiset Sd and the relation ^> may be represented as a directed weighed 
graph, where the nodes of the graph represent the elements from Sd and the arcs 
of the graph represent the relation ~k In this graph we can distinguish two classes 
of nodes - sinks, that is, nodes having only incoming arcs and isolated nodes, 
having neither incoming nor outgoing arcs. 
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2.2 DOMAIN Definition 

The DOMAIN algorithm is presented in Fig[TJ in Fig. [51 we present the behavior 
of the algorithm. 

The input for the method is the examined set of values for a given attribute 
provided as a multiset Sd\ the output is the set D l of the discovered domain 
values and ideally we expect D' AD — 0. The behavior of the algorithm is driven 
by two input parameters: e - the textual similarity threshold and a - the mul- 
tiplicity ratio threshold. 



Procedure calculateSimilarities (Sd, e ,a) 
for all dcSn do 
for all djtSD do 
s^sim (di, d;} 
r • ratio (di, dj) 
if (s>e) a (m(di)<m(dj) ) a (r> a) 
then 

similarities [di, d-j] * (s, r) 
end if 
end for 
end for 
return similarities 

Procedure get Sink (di ,dz , parents , £) 
if ( (parents [dz]= NULL) 
v (parents [da] !- NULL 
a sim (d:, parents [daj ) < £} 
then 

return d?. 
else 

return get Sink (di, parents [d2] , parents, £) 
end if 

Procedure graphToTree (So, similarities) 
for all di€Sc do 

parent [di]- NULL 
end for 
for all d±6SD do 

parent [di ]• arg maxd, similarities [di,dj] 
end for 
return parent 



Procedure streamline (So, parents , e) 
for all di s Su do 

if (parents [d-:] != NULL) 

A ( ! 3dj : parents [dj]=di) 
then 
parents [di] • getSink (di, parents [di] , 
parents, e ) 
end if 
end for 

for all d e Sd do 
if (parents [di] ! =NULL) 

A (3d; : parents [d;]=di) 
then 

parents [d^] — NULL 
end if 
end for 
return parents 

Procedure DOMAIN (Sd ,£,a) 

similar it ies^calculateSimilari ties (Sd, e, a) 

parent*-graphToTree (Sd, similarities) 

parent —streamline (Sd, parent) 

for all dieSD' do 

if parent [d-:]=NULL then 
D'- D'u parent [di] 

end if 
end for 
return D' 



Fig. 1. The pseudocode of the DOMAIN method. 
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Fig. 2. A fragment of the graph structure after the application of the steps of the al- 
gorithm. The sizes of nodes represent the number of value occurrences. In fig. A., we 
created the graph with all the arcs satisfying the conditions of the ~> relationship. In 
Fig.B. we retained only the arcs representing the greatest similarity between the val- 
ues. In Fig.C, the node F. has been transformed into a sink node as the relation ~> 
between A and values J,K,L did not occur. In Fig.D. we streamlined the graph and 
identified values A, B, C and D as the domain values. 
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3 Experimental Results 

We performed the experiments on two datasets - a synthetic 'American cities' 
dataset and a real-life 'Polish cities' dataset. The latter data set consists of 
the names of Polish cities extracted from an Internet survey and contains 2199 
distinct values (28429 in total) of which 1862 were domain values (valid Polish 
city names), covering 27279 elements of the data set. The synthetic 'Ameri- 
can cities' data set was created using the names of 50 of the largest American 
cities as the domain values. The incorrect variants of these values were gener- 
ated according to the distribution of the textual errors in the former data set. 
After these operations, there were 1350 distinct values (24749 total values) and 
the domain values covered 22516 values from the data set. The purpose of the 
experiments was to compare the domain discovery effectiveness of DOMAIN to 
the effectiveness offered by the OWB. 

The effectiveness of domain discovery was assessed using two measures. The 
first measure pn is the ratio of discovered domain values to the all domain values 
in the given dataset; the second measure p n n is the ratio of the non-domain 
values classified as domain values to the all non-domain values in the given data 
set. Both measures are also expressed in an absolute form, where we take into 
consideration the number of occurrences of the value within the data set; defined 
respectively as Pjj and P u d- 



D'C\D\ _ \{ter | t{A) e (D'r\D)}\ 

\D\ " \{ter \t(A) e D}\ 

NUE)\ _\{ter | t{A) e (D'n(N 

nUe\ nD ~ |{ter I t{A) e {N \J E)}\ 



Pd = Pd — 

\D\ |{t€r \t{A) eD}\ 

_\D'n(NUE)\ _ \{t e r \ t(A) e (D' n (N U E))}\ 

PnD — 1 at- i i Fl " nD = ~ . - .- — ^r^ 



3.1 Results 



We performed the experiments on the two aforementioned data sets for e = 0.87 
and a = 2 obtained using the results described in [3] and yielding optimal 
results in terms of the effectiveness measures. For comparison, we ran the Oracle 
Warehouse Builder profiler on the same data sets and chose the optimal results 
obtained using this tool. In our experiments, we chose two sets of the driving 
parameters, denoted respectively by OWB(l) and OWB(2). In both sets, we 
set the limit for the maximal number of discovered values to 10000 to assure 
the discovery of all the domain values. In case of OWB(l) the lower limits were 
set to to allow the discovery of infrequent values; in case of OWB(2), we set 
the lower limits to 1 hence limiting the discovered values only to those whose 
frequency of appearance in the data set was greater than 1%. 

The results of the experiments are presented in Fig. [3l 

The experimental results prove that our approach is capable of effective discov- 
ery of domain values for textual attributes heavily affected (10%) by typographic 
errors. DOMAIN managed to discover all the domain values in the synthetic data 



Discovery of Domain Values for Data Quality Assurance 



19 



American cities 



0.8 ■ 

0.6 - 

0.4 - 

0.2 - 

- 



Pd 



OWB(l) EXX3 

OWB(2) ®zm 

DOMAIN mm 



OWB(l) E 
OWB<2) 1 
DOMAIN I 



1 



Fig. 3. The results of the domain discovery executed using Oracle Warehouse Builder 
(OWB) and our approach on two data sets - synthetic 'American cities (left frame) 
and real- world 'Polish cities' (right frame). 



set and 90% of the domain values in the real-world data set while maintaining 
low level of the ratio of the non-domain values identified as domain values - 1.5% 
of non-domain values that covered 0.8% of relation tuples in case of synthetic 
data and 34% of non-domain values covering less than 20% relation tuples in 
case of the real- world data. 

The OWB was less effective than DOMAIN in terms of the effectiveness mea- 
sures for both data sets. In the case of the synthetic data set and the OWB(2) 
parameter set, the OWB managed to identify 28 of the 50 domain values that 
covered 87% of the relation tuples, whereas our method identified all the do- 
main values. The OWB managed to avoid false positives for the cost of missing 
almost a half of the valid domain values. When applied to the real-world data 
set, the OWB profiler using the OWB (2) parameter set identified only 12 out 
of 1862 domain values which is 0.64% of all the domain values which covered 
57% of the relation tuples. Similarly to the synthetic data set, the ratio of false 
positives was 0. In contrast, our algorithm identified 90% of the domain values 
covering the 99% of tuples. The number of false positives produced by DOMAIN 
covered less than 20% of the tuples. 

In case of the OWB(l) parameter set, the OWB marks all the values from 
the set as domain members producing 100% ratio of false positives. 

Our results show than the DOMAIN method us can be used for the effective 
discovery of domain values in high cardinality data sets heavily affected by data 
quality issues. In contrast, the OWB profiler is more effective false positive- 
wise in situations where the cardinality is low and only takes into consideration 
the coverage of the examined relation by the potential domain values as in case 
of high cardinality data set it can either discover either a fraction of the domain 
values or denote all the values as the domain members. 
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4 Conclusions 

In this paper, we have presented the results of experiments performed using 
the DOMAIN method applied to synthetic and real-world data. The results 
show that DOMAIN can effectively discover domain values with high accuracy 
and a low ratio of false positives in data sets suffering from data quality issues. 
The method has been compared to the commercial solution offered by the Or- 
acle Warehouse Builder and showed greater effectiveness while also maintain- 
ing a higher level of flexibility The presented method can be used to support 
the data profiling stage of the data quality assessment and improvement pro- 
cess via the automatization of work needed to obtain the up-to-date metadata 
describing a given dataset. 
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Abstract. Dynamic time warping(DTW) is widely used for accelero- 
meter-based gesture recognition. The basic learning strategy applied with 
DTW in most cases is instance-based learning, where all the feature 
vectors extracted from labeled training patterns are stored as reference 
patterns for pattern matching. With the brute-force instance-based learn- 
ing, the number of reference patterns for a class increases easily to a 
big number. A smart strategy for generating a small number of good 
reference patterns is needed. We propose to use DTW-based K-Means 
clustering algorithm for the purpose. Initial training is performed by 
brute-force instance-based learning, and then we apply the clustering 
algorithm over the reference patterns per class so that each class is rep- 
resented by 5 ~ 10 reference patterns each of which corresponds to the 
cluster centroid. Experiments were performed on 5200 sample patterns 
of 26 English uppercase alphabets collected from 40 personals using a 
handheld device having a 3-d accelerometer inside. Results showed that 
reducing the number of reference patterns by more than 90% decreased 
the recognition rate only by 5%, while obtaining more than 10-times 
faster classification speed. 

Keywords: Dynamic Time Warping, K-Means, Accelerometer, Gesture 
Recognition. 



1 Introduction 

Nowadays, most mobile devices, including game console controllers and smart 
mobile phones, are designed to include inertial sensors for improved natural user 
experience. The most popular inertial sensor is the MEMS-based accelerometer. 
With accelerometer, it is possible to capture and recognize the posture as well as 
the motion of devices. Several research efforts tried to develop gesture recognition 
systems based on acceleration signals from mobile devices |3|— 15|] . 

In many of such efforts, template matching-based 1-nearest neighbor classifica- 
tion combined with dynamic time warping(DTW) has been successfully applied, 
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where brute-force instance-based learning is used for training in most cases. With 
brute-force instance-based learning, every training pattern is added to the set of 
reference patterns, which can make the size of reference patterns very large. Large 
number of reference patterns incur the problem of low performance and aggressive 
resource consumption that limits the practical value of the system. 

In this paper, we propose a DTW-based clustering algorithm to optimize the 
number of reference patterns in a systematic way. In our method, training a 
classifier is performed in two phases. In the first phase, a classifier is trained 
according to the brute-force instance-based learning. In the second phase, the 
DTW-based K-Means clustering algorithm is run on each gesture class over all 
the reference patterns. The algorithm partitions the reference patterns into a 
small number of clusters according to the similarity measured by DTW and 
extracts the centroid of each cluster. The centroids of the clusters become the 
new optimized reference patterns for each gesture class. One important issue 
in applying K-Means with DTW is that the reference patterns are different in 
length. We cannot simply estimate the centroid of non-uniform length patterns 
as they are in different hyperspaces. The solution we propose is to warp the 
reference patterns along the warping path determined by DTW. By repeating 
pairwise warping across all the patterns in a cluster, it is possible to estimate 
the centroid of the cluster. We describe the details of the algorithm in section 2. 

To show the effectiveness of the algorithm, we applied the algorithm to the 
problem of recognizing 26 English uppercase alphabets written using a handheld 
device having a 3-d accclcrometer. The results from the experiment are presented 
in section 3. 

2 DTW-Based Temporal Pattern Clustering 

In this section, we provide detailed description of how patterns of non-uniform 
length can be clustered based on DTW measure. The basic idea is to average 
two patterns along the warping path generated by DTW. Let us assume that 
P and Q are two patterns, each of which is respectively represented as n x k 
and n x I matrices. By applying DTW, we can get a warping path W , which 
is an ordered list of index pairs. Each index pair (i, j) corresponds to the best 
matching element pairs from P and Q respectively. With W, we can calculate 
the centroid M of P and Q as follows. 

M i = ±(P i + ] lr l J: l fJ 1 F j ),l<i<k 

F = {Qw m , 1 \W rnfl =i,l<m< \W\} 

m is the index of the warping path W. The length of the warping path is \W\. 
W m fi is the index of P's element that lies in the m-th place in the warping 
path, while W m ^\ is the index for Q. Consequently, F represents the set of 
Q's elements that are matched to Pi. Extending the above formula to N pat- 
terns , we get the algorithm^ In the algorithm, P is a list of N patterns, and 
WarpingPathByDTW(n, M, \M\, Pi, |P,|) is assumed to calculate the warping 
path W of two input patterns M and Pi using DTW. 
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Algorithm 1. DTW-MeanForNPattems(n, P, N, M) 

for i = 2 to N do 

W ^WarpingPathByDTW(n, M, \M\,P it \Pt\) 
for j - 1 to \M\ do 

Q^{Pi,w al \W xfi =j,l<x< \W\} 



Mi 
end for 
end for 

M <- iM 



Mi + jh E 



IQI n , 

|Q| Z^fc = l "V= 



The key to the algorithm [5] is the /or-loop where input vectors are incre- 
mentally added along the warping path W. After adding all the N patterns, the 
centroid pattern is calculated simply by dividing At by N. With the algorithm^ 
it is possible to apply K-Means clustering on non-uniform length patterns to op- 
timize the reference patterns for each gesture class. 

3 Experimental Result 

We evaluated the DTW-based K-Means clustering algorithm by applying it on 
a problem of recognizing gesture patterns. 

We used Nintendo™Wii remote to collect gesture pattern data. Wii remote is 
a handheld device that embeds a 3-axis accelerometer. Users performed gestures 
while holding the device by hand. The embedded accelerometer captures the 
hand movements as a series of acceleration changes across the 3-axes of (x, y, z). 

The target patterns for this experiment are uppercase English alphabets, each 
stroke of which is defined as in Figure [TJ 40 adult people participated to perform 
gestures. Each person played every alphabet 5 times, so each alphabet has 200 
pattern samples. The total number of patterns for all the 26 alphabets are 5200. 

The signal preprocessing stage for feature extraction is shown in Figure [2j 
The input to the preprocessing is a train of 3-axis acceleration signal sampled 
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Fig. 1. 26 capitalized English alphabet gestures are collected. 
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Fig. 2. 26 capitalized English alphabet gestures are collected. 

in lOOhz. In the first phase, motion detection is performed for detecting the 
duration in which a gesture is performed. The detection algorithm depends on the 
variation of motion energy inside a sliding window. The details of the algorithm 
is out of the scope of this paper. The output of the motion detection phase is 
a segment of acceleration signals. It is then smoothed for noise reduction using 
1-D Gaussian filter, resampled in 50hz for reducing the number of samples, and 
finally normalized to the values between -5.0 ~ 5.0. 

In our experiment, we used a 1-NN classifier with DTW as the distance mea- 
sure. Training is in two-phases: pure instance-based learning in the first phase 
and then DTW-based K-Means clustering for optimizing the reference patterns 
in the second phase. As we have 5200 sample patterns, applying 10-fold cross 
validation produces a 1-NN classifier with 4680 reference patterns and 520 test 
patterns. Each class has 180 reference patterns after the first phase. We mea- 
sured the performance of the classifier with all the 180 reference patterns, and 
then we optimized the number of reference patterns by applying the DTW-based 
clustering algorithm and compared the performance against the original classi- 
fication performance. We tested with varying number of clusters and the result 
is shown in table [TJ 



Table 1. Recall and precision according to different number of clusters. 
No. of clusters per class 10 20 30 40 50 180 



Recall(%) 



.1 89.6 90.4 90.8 91.2 92.5 



Precision(%) 



:7 90.5 90.9 91.2 92.6 



The recall rate constantly decreases as the number of reference patterns is 
reduced, but the degree of decrease is not so significant. With 10 reference pat- 
terns per class, which is 5% of the non-clustered 180 reference patterns per class, 
the recall rate stays only 4.8% less than that of the non-clustered case. 

Table [5] shows how long it does take to classify a test pattern with different 
number of reference patterns. It is noticeable that in the case of 10 reference 
patterns per class, the classification took only 8% of the time for non-clustered 
case with 180 reference patterns. 
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Table 2. The relation of performance with the number of reference patterns. 

No. of clusters per class 10 20 30 40 50 180 

Time (msec) 51.123 96.730 140.985 184.173 226.155 657.220 



K-Means clustering stops when no cluster center does move anymore. In our 
experiment, about 10% of the clustering cases did not converge to the stable 
status. In such cases, we need another criteria for stopping the clustering process. 
We used two criteria. First, we limited the number of iteration under 100. If a 
clustering process does not stop until it iterates 100 times, the process is forced 
to stop and the final cluster centers are considered to be the results. Second, we 
put a threshold on the minimum distance of center movement. A cluster center 
is flagged as has been moved only when the distance of the movement is greater 
than the threshold. 



4 Related Work 

Leong et al. experimented with a very small number of reference patterns to 
see how DTW performs on accelerometer-based gesture patterns, and showed 
that DTW performed very well with over 90% recognition accuracy |3|. The 
experiment is limited to a set of small number, 10, of very discriminative and 
simple gestures under user-dependent cases only though, and a systematic way 
of choosing good reference patterns was not given. 

Liu et al. proposed an adaptation algorithm to limit the number of reference 
patterns [4j]. Their method initially assigns only two reference patterns per ges- 
ture class. On receiving a new training sample, an evaluation is performed on 
previously selected reference patterns as well as on the new training sample, and 
one of them is discarded. 

Bong et al. utilized K-Means clustering to partition reference patterns for 
each gesture class into a small number of clusters and use the centroid of each 
cluster as the reference patterns, which is very similar to our method. One major 
difference is that Bong et al.'s K-Means clustering estimates the centroid of a 
cluster by resizing reference patterns into an average length so that the patterns 
become of a uniform length. 

Chen et al. proposed a method to utilize DTW for fuzzy clustering. The use 
of DTW is limited to calculate pairwise distances of input patterns to build a 
distance matrix as an input to fuzzy clustering algorithm. DTW is not used 
at runtime for classify ingtest patterns. Their specific algorithm runs in the 
complexity of 0(2 n — 1) |l|. Our method is different in that we use DTW both 
as a distance measure and as a tool for deriving cluster centers. Also, at runtime, 
DTW-based simple 1-NN algorithm is enough for classifying test patterns. 

Oates et al. introduced time series clustering methods that are based on DTW 
and HMM. They utilized DTW not only as a distance measure but also as a 
method for finding cluster centers. To find the center of a cluster, their algorithm 
first selects a pattern that minimizes distance to all other patterns in the cluster. 
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Then non-selected patterns are warped into the selected pattern. Then a simple 
averaging is used to derive the center pattern [2J]. Their approach is similar to 
ours in that DTW is utilized for calculating cluster centers. But they did not 
clearly show the details of their method. We present details of our method and 
algorithms for deriving cluster centers using DTW. 

5 Conclusion 

Clustering temporal patterns is not easy because temporal patterns are non- 
uniform in length and discriminating features are not temporally aligned. We 
proposed an algorithm for calculating centers of temporal patterns based on 
DTW, which enables us to implement K-Means clustering for temporal patterns. 
We validated the performance of the algorithm by utilizing it for optimizing the 
number of reference patterns of a 1-NN classifier that learns by instance-based 
learning. We observed, in our experiment, that reducing the number of reference 
patterns down to 5% using the proposed clustering algorithm did deteriorate the 
recognition performance of the classifier only marginally. On the other hand, af- 
ter reducing the number of reference patterns, the classifier performed more than 
10 times faster, which is a critical benefit to utilize such classifiers in resource- 
bounded platforms. 
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Abstract. Post-processing of association rules with interestingness 
measures is considered as one of the most difficult and interesting task in 
the research domain of Knowledge Discovery from Databases (KDD). In 
this paper, we propose a new approach to discover the behaviors of in- 
terestingness measures by modeling the interaction between them. The 
interaction values are calculated based on the capacity function (also 
called fuzzy measure) and then are segmented to discover the interac- 
tion's trends of clusters of interestingness measures. 

Keywords: Knowledge Discovery from Databases, association rules, in- 
terestingness measures, correlation graph, capacity function, Sugeno 
measure, interaction value, segmentation. 

1 Introduction 

The analysis of the relations between interestingness measures based on the cal- 
culations of interestingness values from a data set structured by rules, especially 
association rules [T], is a challenge in Knowledge Discovery in Databases (KDD). 
Each interestingness measure is often considered as representing an aspect or a 
property on the data set studied. The process of determining the approriated 
interestingness measures for a data set always a complex problem to resolve. In 
addition, the use of only one interestingness measure to reflect the whole data 
set is very difficult. In reality, each data set has itself many aspects that are not 
express obviously. 

In this paper, we deal with a new approach based on the interaction between 
interestingness measures [3] [1] [7] , using the a graph model called interaction 
graph. After the determination of clusters of interestingness measures and the 
selection of the approriate interestingness measures that are strongtly represent 
the specific aspects of the data set, we can easily rank the association rules 
generated from the data set according to the order of the common interestingness 
measure obtained from the group of interestingness measures chosen. 

2 Interestingness Measures 

Suppose that we have a data set D contains the transactions in a supermarket 
[1]. The set of items (itemset) / = {i\ 1 i%, ..., i n } corresponds with a set of avail- 
able articles in a supermarket. The set X, contains the articles appeared in a 
transaction. In fact, the transaction expresses the bought articles of custormers. 
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An association rule is represented in the form X ^ Y(X P\Y = $) [T] with X 
itemset and Y itcmset are generated by the algorithm APRIORI pQ. 

The interestingness value of an association rule is calculated via a function 
m with the parameters n,nx,n Vl n x y (see FigfT]). In this case, the funtion m 
is called an interestingness measure m{X — > Y) — f(n,nx,nY,n x y) on asso- 
ciation rules. The importance of an association rule depends strongly on the 
interestingness value that it can obtain from the corresponding interestingness 
measure. 




Fig. 1. Venn diagram for an association ruleX — » Y. 

3 Interaction between Interestingness Measures 

3.1 Interaction Value 

The interaction value between two interestingness measures is calculated by a 
capacity function (i.e., a fuzzy measure, a measure of capacity) [2] [5] @]. The 
capacity function ^ on a set of interestingness measures \i : 2 2 — > [0, 1] with 
the following axioms [2] [9]: 

— //(0) = and /x(Q) = 1 (normalization). 

— fi(A) < /J.(B) when A C B C Q (monotonicity) 

The value fJ,(A) demonstrates the capacity of a subset A of interestingness mea- 
sures. A capacity function Sugeno [5] or called a capacity with the parameter A 
(A-measure) , if exists a value A > — 1 so that : 

H(A U B) = fi(A) + n{B) + Xn(A)n(B) (1) 

with A C QandA C Q. 

The capacity function Sugeno can be determined by the method "Singleton 
Fuzzy Measure Ratio Standard" [10]. When the function /i does not carry on 
the additive property (/J,(A U B) ^ fJ-(A) + fi(B)) we have : 

1 + A = IJ(1 + \n( Xi )) (2) 

i=l 
with : q = \Q\. 
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3.2 Interaction Matrix 



Each intercstingness measure is studied the interaction relations with the q — 1 
remaining interestingness measures. In our work, we only focus on the interac- 
tion relations from each couple of interestingness measures irii,mj so that the 
interaction value £y between two interestingness measures is peformed by the 
formula [7] |TT]: 

& = ^({m t , mj}) - fJ.({m t }) - n({mj}) (3) 

A remark is that the interaction value^- between to interestingness measures m, 
and rrij has the symmetric property £y = ^ . After calculating the interaction 
values between each couple of interestingness measures, we have a matrix of 
interaction values £ (i.e., interaction matrix): 

£n 6.2 ••• £lq 

£21 £22 ••• £,2q 
£gl £<j2 ■■■ £qq 

3.3 Interaction Graph 

For an intuitive observation of the interactive relations obtained from an inter- 
action matrix, we have proposed a graph model (i.e., interaction graph). The 
interaction graph is an extension of the correlation graph [5] [llj . 
In the interaction graph, G — (V, E) 

— V : each vertex represents an interestingness measure. 

— E : each edge is the interaction value between the two vertices 

An interaction graph with q interestingness measures will have 2 edges. It 
is difficult to observe and to evaluate the interaction relations, especially the 
strongly interaction relations. The interaction relation exists when the interac- 
tion value is greater or equal a predetermined threshold r , called r-interaction. 



3.4 Tool for Interaction Graph 

The model of interaction graph is implemented in the INTERACTION module of 
the ARQ AT tool [6] . This module also provide the important means to calculate 
the interaction values between the interestingness measures via the capacity 
function Sugeno. The module performs in the following steps : 

1. Determine the first initialized values of the capacity function for each in- 
terestingness measure from the data set or the user's points of view by the 
submodule CAPACITYINITIALISATION. 

2. Calculate the values of the capacity function with the algorithm "Singleton 
Fuzzy Measure Ratio Standard" [TD] by using the submodule CAPACITY- 
COMPUTATION. 
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3. Perform the calculation of the interaction values and establish the interaction 
matrix by the submodule INTERACTIONCOMPUTATION. 

4. Build and visualize the interaction graph by the submodule INTERAC- 
TIONVIEW. 



4 Experiments 

4.1 Values of Capacity Function 

The initial values of capacity function of each interestingness measure is de- 
termined by the ratio between the number of different interestingness values 
obtained and the total number of association of a rule set. (see table [1J. 



Table 1. Initially capacity values of interestingness measures. 



Interestingness 
measures 


Initially capacity 
values 




Interestingness mea- 
sures 


Initially capacity 
values 


Conviction 


0.54 




Confidence 


0.002 


EII (a = 1) 


0.604 




Dependency 


0.062 


EII (a = 2) 


0.536 




Phi-Coefficient 


0.014 


F-measure 


0.016 




Rule Interest 


0.438 


Implication index 


0.704 




Support 


0.086 













With the initially capacity values in the table [TJ we can calculate the capacity 
value of Sugeno for each interestingness measure by the method " Singleton Fuzzy 
Measure Ratio Standard" [TU] (see table |2J). Table[5]describes in detail the values 
of the Sugeno's capacity function obtained from the rule set. 



Table 2. The capacity value of Sugeno for each interestingness measure. 



Interestingness measures 


Sugeno's 

capacity 

values 




Interestingness measures 


Sugeno's 
capacity 
values 


Causal Confidence 


0.0002 




J-measure 


0.0384 


Causal Confirm 


0.0020 




Kappa 


0.0017 


Collective Strength 


0.0082 




Least Contradiction 


0.0037 


Confidence 


0.0002 




Lerman 


0.0037 
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4.2 Interaction Values 

A part of interaction values obtained from interestingness measures on the rule 
set is represented in table |3] We can see that the measures EII and Implication 
Index has the highest value of interaction (the interaction value is 1). 

Table 3. The capacity value of Sugeno for each interestingness measure. 



Interestingness measures 1 


Interestingness measures 2 


Interaction values 


EII 


Implication Index 


1 


Conviction 


Implication Index 


0.892268317 


EII 2 


Implication Index 


0.885549175 


Conviction 


EII 


0.763155019 


EII 


EII 2 


0.757408143 


Implication index 


Rule Interest 


0.721446156 









4.3 Interaction Graph 

We can observe rank of interaction on an interaction matrix based on the deter- 
mination of different values of the threshold -interaction. For instance, to observe 
the different levels of strong interaction, we can start from the couple of inter- 
estingness measures having the the strongly interaction values and the other 
couples of interestingness measures corresponding to the decrease of interaction 
values (Fig. |3 ). 



|f1B) Irnnllcaton Index] 




|"(7}C&m*1clnin~| 
Fig. 2. An interaction graph. 



On Fig. [3 we can observe the affect the measure Implication Index with regard 
to the other interestingness measures in a cluster of 5 interstingness measures 
Implication Index, EII, Conviction, EII 2, Rule Interest. The number of inter- 
estingness measures in a cluster increases according to the decrease of the value 
of r-interaction. 
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5 Conclusion 

We can evaluate the behaviors of interestingness measures by segmenting the 
set of interestingness measures in the interaction graph. By using the capacity 
function of Sugcno (corresponding to the parameter value of A = 0.5 ), at the 
first step, we have modeled the interaction between 40 interestingness measures 
via the interaction matrix and the interaction graph. The observation of the 
interaction between interestingness measures can be performed via the decrease 
of interaction values (i.e., the threshold r- interaction) or the segmentation of 
interaction values. Based on the clusters of intcrstingness measures found, the 
user can select the appropriated clusters of interestingness measures to evaluate 
the quality of knwoledge represented in the form of association rules. 
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Abstract. This paper presents the architecture of the Intelligent Learning 
Systems for training of power systems operators and describes one of its 
components: the tutor module. Operators can acquire knowledge in different 
ways or with different paths of learning, also called models of sequence. The 
tutor model is an adaptive intelligent system that selects the sequence of the 
learning material presented to each operator. The adaptive sequence is 
represented as a decision network that selects the best pedagogical action for 
each specific operator. The decision network represents information about the 
current state of the tutor, their possible actions, the state resulting from the 
action of the tutor and the usefulness of the resulting state. The model was 
evaluated using graduate students with good results. Based on the adaptive 
model, we developed an Intelligent Learning System called as SI-Aprende. The 
SI-Aprende system manages, spreads and promotes the knowledge by mean of 
the search and recovery of SCORM Learning Objects. 

Keywords: adaptive learning, intelligent environment, learning objects, 
SCORM, sequencing model. 



1 Introduction 

Training of operators is an important problem faced by electrical power plants. The 
process of learning how to control, maintain and diagnose in a complex industrial 
environment takes years of practice and training. An operator must comprehend the 
physical operation of the process and must be skilled in handling a number of 
abnormal operating problems and emergencies. The problem increases when the 
complexity of the system obstructs the efficiency, reliability and safe operation. 
Therefore, the novice and experienced operators need continuous training in order to 
deal reliably with uncommon situations. 

Most of the learning proposals of intelligent systems in education are based on 
Intelligent Tutoring Systems (ITS) [1], which are interactive learning environments 
that have the ability to adapt to a specific student during the teaching process. 

K.G. Mehrotra et al. (Eds.): Developing Concepts in Applied Intelligence, SCI 363, pp. 33-p^. 
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This paper describes the architecture and development of an intelligent environ- 
ment (IE) for training of power systems operators. The IE takes elements of artificial 
intelligence such as student modeling, intelligent tutors, intelligent sequence, concept 
maps and reusable Learning Objects (LO) to build a tool that allows meeting the re- 
quirements for learning and training of power system operators. In contrast with a 
traditional training system, the main goal of the intelligent environment is to certify 
operators in knowledge, skills, expertise, abilities and attitudes for operation of power 
systems. 



2 Architecture of the Intelligent Learning System SI-Aprende 

The architecture of the IE is based on the generation of dynamic courses generating 
systems proposed by Brusilovsky [2]. The IE is composed of four main components 
(see Figure 1): the domain knowledge module, the tutor, the operator model, and the 
learning management system (LMS). 

The first module contains the knowledge of the domain as LOs and concept 
structure maps. The tutor module is the component that generates the intelligent 
sequence of learning objects to be presented to the operator as a course. Taking as a 
basis the concept map, the pedagogical component and the operator model, the 
sequencing model generates a specific course for each operator. The operator model is 
used to adapt the IE to each specific operator and this model is divided in three 
subcomponents: cognitive component, operator profile and affective component. 
Finally, the LMS controls the interactions with the operator, including the dialogue 
and the screen layout. 



Operator 
model 



Cognitive 



Tutor 



Domain 
knowledge 




Fig. 1. Architecture of the intelligent environment (adapted of [3]). 



This paper describes in detail one of the components of the IE, the Tutor Model. 
Detail description of the Operator Model, Domain Knowledge Model and LMS are 
already described in a previous work [4]. 



SI-APRENDE: An Intelligent Learning System 35 

3 Tutor Module 

The main component of the tutor module, also known as instructional or educational 
model, is the sequencing model that contains the sequence of LOs to be presented to 
an operator as a self instructional nontraditional course. The adaptive sequence is 
represented as a decision network that selects the best pedagogical action for each 
specific operator. The goal is to present to the operator the learning materials that 
better fits his learning needs. 

A decision network or influence diagram represents information about the 
current state of the tutor, their possible actions, the state resulting from the action of 
the tutor and the usefulness of the resulting state. It can be seen as an extension to 
Bayesian Networks with random nodes, incorporating decision nodes and utility 
nodes. The sequencing model consists of the following random variables (see Fig. 3): 
knowledge of LO, satisfaction of the objective, progress of the objective, progress of 
the activity, score of LO, quiz, project, task, practice. 




Fig. 2. Decision Network. 

As shown in the figure above relations between the nodes are causal. KwLO random 
node has four values: Very Good, Good, Enough and Not Enough. The decision node 
considers the pedagogical actions that will be evaluated according to the utility 
function to select the best one, and in this work four possible pedagogical actions are 
identified: LO challenge, Next LO, Repeat refined and Repetition of LO. The 
calibration of the decision network is given by the experts in the domain. The utility 
table entries are set based on the teacher's experience about the best over all possible 
pedagogical actions according to the given knowledge state and the student's 
interaction. Adding the evidence of the previous Knowledge of the LO's random 
node, as well as the previous pedagogical action the DN was turned into a dynamic 
decision network (DDN). 
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4 Model Validation 

A model validation is been carried out in two stages; the first one was evaluated in an 
academic environment, and the second one is running in a business environment pro- 
posed in this work. We have not finished last, because the training course of the Elec- 
trical Power Plants operators is too long and the operators are still using the Intelli- 
gent Environment. 

Following, the first evaluation process will be described it. In the first stage, three 
groups of summer students were selected at the graduate degrees from Tecnologico de 
Monterrey, Mexico City campus. The courses were taught by different instructors, 
with a total population of 58 students (N = 58) distributed in Table 1 as follows: 

Table 1. Groups of Students. 



Course 




# Students 




Focus Group 


Control Group 


Math II 

Electricity and Magnetism 

Introduction to Physics 


15 

9 

5 

NFocus= 29 


15 

9 

5 

NControl= 29 



The Focus Group of students used the learning objects of IE while the Control 
Group used the traditional planning course. The LOs were designed as additional 
support for the students to have more tasks and exercises that they can perform 
anywhere, anytime. Three focus groups were heterogeneous in the level of subject 
knowledge, skills and even interest, since they were formed by students from different 
disciplines and professional careers. The time period in which the system was used 
was 15 days, having unlimited attempts at each learning activity and varied resources, 
from static HTML pages or text documents to simulations and videos. 

The starting point was a diagnostic evaluation (PreTest) of the topic to all students 
of the course on a scale of 0-100. After applying the pre-test, the instructors made the 
separation of focus and control groups, giving appropriate details in both cases, 
indicating the time that their learning activities would take. The material prepared by 
instructors consisted of tasks, projects, practices and self-assessments of a specific 
topic of the course. The learning objective was the same for both groups, but the way 
of reaching the goal was different for each group. 

The instructors created four LOs about a specific topic of the course and all 
students in the focus group started from the same LO. If the student received very 
good grades (or equivalent numeric 9 <= 10 on a scale of 1 to 10) in the first LO, it 
was designed a greater challenge or a higher level of difficulty for that student, 
seeking to have material that fits the needs and learning pace of each student. If a 
student got a good grade (or its equivalent numeric 7 <= 9 on a scale of 1 to 10) it was 
designed a LO with the same level of difficulty of the first LO. If the student had 
enough grade (or equivalent number 6 <= 7 on a scale of 1 to 10) it was designed a 
LO with lower level of difficulty that the student may reach the desired learning 
objective or could even repeat the same LO. If the rating was not enough for the 
student (or its numeric equivalent 1 <= 6 on a scale of 1 to 10) the student got the less 
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difficult LO. This design was established by the instructors but the pedagogical action 
that takes the system is the result of the propagation of a dynamic decision network 
that considers the knowledge of the LO and other variables, such as the progress of 
the objective, progress of learning activity and the satisfaction of the objectives for 
each student. 

After two weeks of using the system, the instructors applied a final evaluation 
(PostTest) to both groups on the subject of pre-test, obtaining the necessary data to 
calculate the learning gain in both groups. According to Hake [5], the following 
formulas were applied to determine the learning gain in both groups: 

it Average pre-Test 

D , £ Pre, 
< **» >= ~^~ (1) 

4> Average post -Test 



< Post >- 



EiPosti 



W (2) 

it Average Relative Gain 

< Post > —< Pre > 

10Q-< Pre > (3) 

^ Simple Gain 

C- = < Post > -< Pr° > (4) 

We summarize our results in Table 2, where we show the average values for the pre- 
Test, the post-Test, the Relative gain and the Simple gain, for the focus and control 
groups of each course. 

Table 2. Results of Control and Focus Group. 



Group 


Pre-Test 


Post-Test (2) 


G 


Grel 




(1) 




(4) 


(3) 


Math II Focus 


17.5 


90 


72.5 


0.88 


Math II Control 


10.5 


71.63 


61.13 


0.68 


Electricity and Magnetism Focus 


8.33 


74.67 


66.34 


0.72 


Electricity and Magnetism Control 


10 


69.11 


59.11 


0.66 


Introduction to Physics Focus 


31.25 


79 


47.75 


0.69 


Introduction to Physics Control 


27.5 


55.33 


27.83 


0.37 



As it was shown in the tables, students using the IE had higher relative and simple 
learning gains as compared to those students that did not used it. 

Although the electrical operators' evaluation process has not finished yet, the qua- 
litative opinions show that SI-APRENDE provides a more dynamic and interactive 
training to the Operators of Power Plants, where the employees really experience the 
acquisition and transfer of skills and knowledge. 
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5 Implementation of the Intelligent Learning System SI-Aprende 

In order to have an environment with a suitable training to certify personnel in 
knowledge, skill, expertise, abilities and attitudes for operation of power systems, we 
developed an Intelligent Learning System called SI-Aprende [6]. 

The SI-Aprende system manages, spreads and promotes the knowledge of the 
power utility, by mean of the search and recovery of SCORM LOs. SI-Aprende 
composed by computer based training components labeled as reusable learning 
objects; a learning object repository; a tutor module based on decision networks to 
select the best pedagogical action for each operator. 

6 Conclusions and Future Work 

In this paper we have presented the architecture of an Intelligent Environment for 
training of Power Systems Operators. One of the components was described: the 
Tutor Module, which contains the sequencing model. An empirical evaluation of the 
sequencing model is presented using graduate students with good results. Based on 
the conceptual model of the IE, we developed an Intelligent Learning System called 
as SI-Aprende. SI-Aprende composed by computer based training components 
labeled as reusable learning objects; a learning object repository; a tutor module based 
on decision networks to select the best pedagogical action for each operator. The 
intelligent tutor module will be a dynamic course generating system interacting with 
the Operator Model to produce specific training material for each operator. Currently 
SI-Aprende has been loaded with 130 learning objects for electric power operators 
training. 
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Abstract. A new LS-SVM method for a multiple-input multiple-output 
(MIMO) channel prediction is presented. A least squares support vector 
machine (LS-SVM) is proposed as a prediction technique. The LS-SVM 
has nice properties in that the algorithm implements nonlinear decision 
regions, converges to minimum mean squared error solutions, and can be 
implemented adaptively. We also formulate a recursive implementation 
of the LS-SVM for channel prediction in the MIMO system. The perfor- 
mance of the new method is shown by a simulation of the bit error rate 
in the given environment. 

Keywords: channel estimation, multiple-input multiple-output (MIMO) 
channel, least squares support vector machine (LS-SVM). 



1 Introduction 

During the last several years support vector machines (SVM) and kernel methods 
[J] > |10j have generated considerable interest and have been used in many appli- 
cations from image processing to analyzing DNA data. These methods belong to 
one of the preeminent machine learning statistical models for classification and 
regression developed over the past decade. 

The multiple-input multiple-output (MIMO) communication systems have 
recently drawn considerable attention in the area of wireless communications. 
Multiple antenna systems are one of the key technologies for the next genera- 
tion wireless communications, especially in WCDMA-based 3G systems |5_. This 
technique has attracted much attention due to its advantage in capacity as well 
as the ability to support multiple users simultaneously 3] . In the MIMO system 
it is usually assumed that the channel state information (CSI) is known at the 
receiver, but not at the transmitter, and the transmitter has no knowledge about 
the channel coefficients. Although the increasing number of antennas improves 
the capacity of the system, it increases its complexity. However, these methods 
waste time while learning the channel when meaningful data can be sent. 

The use of the SVM method and recently developed the least squares (LS) 
SVM method [7] have been proposed to solve a number of digital communication 
problems. Among others, signal equalization and detection for a multicarrier 
(MC)-CDMA system based on SVM linear classification has been investigated by 
Rahman et al. [I]. In the paper by Sanchez-Fernandez et al. [5] SVM techniques 

K.G. Mehrotra et al. (Eds.): Developing Concepts in Applied Intelligence, SCI 363, pp. 39-|44|. 
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have been used for a nonlinear channel estimation for multiple-input multiple- 
output systems. The SVM technique for a robust channel estimation in the 
OFDM data structure was proposed by M.J. Fernandez-Getino Garcia et al. in 
the paper [2J. However, none of these solutions can be used easily in on-line 
adaptive algorithms. 

In this paper, we propose a LS-SVM solution for channel prediction. There 
are two novelties in our proposal. We use the LS-SVM for channel prediction in 
a MIMO system in which inequality constraints are replaced by equality con- 
straints and a quadratic error criterion is used. 

This paper is organized as follows. Section 2 presents the system description 
and the problem formulation. Section 3 introduces a recursive least squares al- 
gorithm that can be implemented on-line. In section 4, the proposed algorithm 
is then evaluated by means of computer simulations. Section 5 presents some 
final conclusions. 



2 The System Description and Problem Formulation 

In this section, the MIMO received model for an un-coded and a beam-forming 
system is introduced. 

In the first case, we assume a MIMO wireless flat fading communication sys- 
tem with 7V r receive antennas and N t transmit antennas modelled by 

y = Hx + n (1) 

where y is the N r X 1 received vector, x is the Nt X 1 transmitted symbol vector 
with each Xi belonging to constelation C with symbol energy E s , and n is the 
while noise vector of size JV r xl with m ~ CA/"(0, No). The channel state matrix 
H = {h mn \ gives a complex channel gain between the m-th receiver and the 
n-th transmit antenna. 

In the second case, the received symbols are expressed in two scenarios, when 
the transmitter and the receiver have full channel information (CSI) and when 
they have the prediction matrix. Then, the channel matrix H = U • D • V 
is a singular value decomposition (SVD) where U and V are unitary matrices 
corresponding to the i-th non-zero singular value an(i), (cij(l) < • • • < &h(M)) 
and M = rank(H). Assuming that x — V\ ■ x we can obtain from Eq. (1) the 
received symbols uf as follows: 

u?y = a H {l)x + u?n (2) 

Assuming u = uf^n we can obtain 

E | h \ 2 = N r ■ No (3) 

Thus, the channel within a MIMO system is time-varying and can be expressed 
in a matrix notation as H = H + E. Therefore, the received symbols are as 
follows: 
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«f y = {°h (1) + Mf J B{) 1 )x + ufn (4) 

In the general case the doppler spread of the signal is greater than the pulse 
bandwidth. In a typical environment the MIMO channel is fast-fading. Assuming 
a MIMO flat fast-fading transmission each sub-channel can be formulated as 
follows |11) : 

hmn\t} = hm n \k) + jh mn {k) (5) 

where in-phase component is represent as 

M 



pi 

Knnik) = ]Jj-f^2cos(2Trf d ksm(a n ) + <2>„ 



(6) 



(?) 



n=l 

and the quadrature component can be written as 

FT M 

h%,„(k) = a/tt J2 cos(27r/ d fcsin(a n ) + & n 

n=l 

where a„ = 27r " 4 ^ +e and $ n ,&n, & are [/[-• 7r,7r]. 

3 The Multidimensional Recurrent LS-SVM 

The recurrent LS-SVM based on the sum squared error (SSE) to deal with 
the function approximation and prediction has been proposed by Suykens and 
Vandewalle [3]. However, the so defined recurrent LS-SVM will not be adequate 
for the channel prediction in the MIMO system. It is caused by the lack of the 
high-dimensional reconstructed embedding phase space. 

In order to extend the recurrent least squares vector machine to a multidi- 
mensional recurrent LS-SVM we introduce scalar time series {si, S2, ■ ■ ■ , st} in 
the form 

h = f(h-i), k = m',m' + l,...,N' + m'-l (8) 

where to', N' are referred to the embedding dimension and the number of train- 
ing data, respectively. 

The function approximation is given by 

s fe = w T (^(s fc _i) +6, k = m',m' + 1,...,N' + m' - 1, i = 1, 2, . . . , ml (9) 

where w = [w\, u>2, ■ ■ ■ , w m '] is the output weight vector, b G R is the bias term 
</>(.) is the nonlinear mapping function estimated by means of using training 
data. 

The recurrent LS-SVM can be formulated as the quadratic optimization 
problem: 

_. 771' N -fm — 1 m 

w min J{m, K efc.i) = ^Yl w i Wi + 2 XI XI e M ( 10 ) 

i— 1 k — m' + l 2—1 
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subject to the following constraints: 

s k e k,i = Wi4>i{sk-i -efc_i) +61 
4 -efe, 2 = wlfoisk-i -efc_i) +62 

s™ - efe, m ' = w^&isk-i - efe_i) + 6 m < 

where fc = m' + 1, m' = 2, . . . , TV' + m' — 1, e& = Sfc — Sfc. Generally, the error 
term here is defined as 

efe = Xfc - f(xk-i) (12) 

4 Experimental Results 

In order to test the performance of the MIMO channel prediction, we used the 
received signal-to-noise ratio (SNR) in the general form 



el 



(13) 



2 



where a^. is the average received signal power, a^ is the predictive error, a. 
is the average noise variance. Thus, after some algebraic manipulations for the 
un-coded system we can obtain 

A E || Hx [[J 

Puc ~ E || Ex \\l +E || n HI (i4j 

and after several manipulations 



Ef =1 ^IW] + ^ 



(15) 



where iV = ranfc(E), (Tjj(j) and <7e(*) are the i-ih non-zero singular values of H 
and E, respectively. 

The MIMO beam-forming can be formulated as follows 

u?y = (<7i + «?)» + n (16) 

Then, after a similar technique as the one used before, we can state the received 
SNR for the MIMO beam-forming system 

„ Ml n 7) 

Pbf E I fiHUDVHf, - a max |2 +^ ^ ( > 

Thus, comparing the above equation with the Eq. (19) we get the value of the 
prediction error a\ for the beam-forming prediction, namely 

a\ = E || Ex |||= E || U H EVx \\\ (18) 



Least Squares Support Vector Machines for Channel Prediction 43 

10" 1 



10" 






io _ 



10" 



10" 



io~ 




o 



20 



25 



5 10 15 

E s /No 
Fig. 1. Bit error rate (BER) as a function of E b /Eq for a un-coded 4x4 MIMO system 
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or after some algebraic manipulations 

a\ = E | (uf UDV H vx - & max )x | 2 (19) 

The RBF kernel with the width parameters: a = 0.5 and the regularizotion 
parameter 7 = 500 were selected, respectively. 

The obtained values of the maximum singular value of the error matrix and 
the minimum singular value of the error matrix are given in Fig. 1 and Fig. 2, 
respectively. 

5 Conclusion 

A recurrent LS-SVM method was used in order to predict a MIMO channel. The 
received SNR for an un-coded and beam-forming MIMO system was derived. 
The proposed solution does not need to use the analytic mathematical model 
of performance measures. However, it can provide the parameters of the system 
in the varying time horizon. The experiment results show that the proposed 
method achieves the best performance for the RBF kernel function. This kernel 
function demands the use of scaling factors for all input parameters. 
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Abstract. Previous nCRF models are mainly based on fixed RF whose dynamic 
characteristics are not taken into account. In this paper, we establish a 
multilayer neural computation model with feedback for the basic structure of 
nCRF. In our model, GC's RF can dynamically and self-adaptively adjust its 
size according to stimulus properties. RF becomes smaller in local areas where 
the image details need distinguishing and larger where the image information 
have no obvious difference. The experimental results fully reflect the dynamic 
characteristics of GC's RF. Among adjacent areas in an image, similar ones are 
integrated together and represented by a larger RF, while dissimilar ones are 
separated and represented by several smaller RFs. Such a biology-inspired 
neural computation model is a reliable approach for image segmentation and 
clustering integration. 

Keywords: Non-classical receptive field, Neural network, Image 
representation. 



1 Introduction 

Machine vision is of great significance to intelligent computing systems. However, 
the vast majority of machine vision researchers overlooked the principle of visual 
physiology. They proposed various algorithms from the engineering point of view. 
Therefore there is a lack of unitive and effective machine vision algorithms to solve 
the problem of complex scenes segmentation. In contrast, the human vision system 
has a strong image processing ability. Therefore, it is entirely possible that we find 
some new breakthroughs to solve the difficulties encountered in computer vision by 
applying the theories of human vision, neuropsychology and cognitive psychology. In 
the human vision system, the retina is also known as "peripheral brain". It processes 
visual information preliminarily. The ganglion cell (GC) is the final place of retinal 
information processing. The receptive field (RF) is the basic structural and functional 
units for information processing in visual system. GC's RF determines the processing 
result and response characteristic of the retinal complex network. Therefore, RF is 
instructive for visual algorithm to simulate the mechanism of information processing 
in the GC. 
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Each visual neuron only elicits response to the stimulation in a specific area of the 
retina (or visual filed). The area is called as the classical receptive field (CRF) of the 
neuron. The CRF of the GC has an antagonistic center-surround structure. With the 
development of in-depth studies on human vision system, many researchers [1, 2] 
found that there is still a large area outside the CRF. Stimulating such area alone fails 
to elicit neural responses but can modulate the neural response to CRF stimulation. 
The area is called as non-classical receptive field (nCRF). Sun C. et al. [3] found that 
stimulating CRF alone can also elicit neural response when the nCRF receive large 
area and appropriate stimuli. Disinhibitory nCRF helps to transmit the luminance and 
grads of image local area [4]. 

Some models were built to stimulate disinhibitory nCRF. Li Z. et al. [5] proposed a 
function with the shape of a volcano whose center is concave. Ghosh K. et al. [6] 
proposed a linear function of three-Gaussian and explained low-level brightness- 
contrast illusions. Qiu F. T. et al. [7] gave a model for the mechanism of mutual 
inhibition within disinhibitory nCRF. These models were established by simple 
reference to the mechanism of nCRF. They were just used in contrast enhancement, 
edge extraction, etc. and had no top-down feedbacks. However the modulations from 
high-level are very important in the neurobiology, which has been proved by 
electrophysiology, anatomy and morphology. 

Research shows that ganglion RF can dynamically change its size [8]. The size of 
RF varies with the changes of brightness, background, length of time of stimulation, 
speed of moving objects and so on. Previous nCRF models are mainly based on fixed 
RF, whose dynamic characteristics are not taken into account. 

Our computational model is not for segmentation. Simulating biological visual 
system needs to address three fundamental issues: first, which features to 
automatically choose for representation; second, what to use to achieve 
representation; third, can biological nervous systems achieve this task? To make a 
computational model solve these three problems simultaneously is very challenging 
and far exceed the scope of the classical segmentation. By establishing ganglion cell's 
dynamic receptive fields and its self-adaptive mechanism, we realized automatic 
analysis and extraction of block features. By simulating the array of ganglion cells we 
achieved representation. And the neural circuit we developed is supported by the 
neurobiological evidence. Therefore, this paper is aimed at how to self-adaptively 
extract features and form internal representations for images by a bio-inspired 
structure. Limited to the length of the paper, we just presented a limited number of 
experimental results. 

2 Physiological Basis 

2.1 Neurophysiological Mechanism of nCRF of Retinal GC 

Neurophysiological studies show that [9] the horizontal cell (HC) with a wide range 
of electrical synaptic connections can receive inputs from large areas of retinal 
receptor cells. The HC affects the activities of receptor cells (RCs) by means of 
spatial summation and feedback and then elicits the GC responses to the stimuli 
within the disinhibitory nCRF through bipolar cells (BCs). Moreover, the amacrine 
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cell (AC) connects many of the nearby ganglion cells (GCs) in the horizontal 
direction through its extensive dendritic branches. Amacrine cells (ACs) also 
interconnect with each other. The breadth of ACs connection goes far beyond GCs 
CRF surround, So AC is properly related to the formation of nCRF with a wide range. 
Thus, HC and AC play the role of information integration in the outer and inner 
plexiform layer respectively. Retinal GC receive inputs from many neurons in the 
outer and inner plexiform layer, hence HC and AC are connected with the formation 
of nCRF of retinal GC. 

2.2 Reverse Control Mechanism of Retina 

The IPC sends messages to HC and BC in the outer plexiform [10]. The works of 
Yang et al. [11] show that IPC ceasing activity in light enhances HC activity. After a 
long period of dark adaptation, IPC accelerates activity, inhibits HC and decreases the 
peripheral antagonist. In addition, some fibers from mesencephalic center (MC) are 
found into the centrifugal fibers of the retina through optic nerves in all vertebrates. 
The fibers from MC control AC and IC. Electrical stimulation to centrifugal fibers 
can increase response of GC to lights. It is possibly caused by the disinhibition of IPC 
and AC. This shows that IPC takes backward control of HC and BC to change the size 
of CRF surround, and MC takes backward control of IPC and AC to change the size 
of CRF center through centrifugal fibers. 

3 Calculation Model 

3.1 Neural Circuit for Dynamically Adjusting of RF 

Based on the above neurophysiological mechanism of retinal ganglion cell nCRF, the 
reverse control mechanism and fixation eye movement, the neural circuit showed in 
Fig. 1 is designed to adjust RF dynamically. The details of retinal micronetwork are 
very complex and partly unclear. So we simplify the RF mode appropriately 
simplification by omitting the contribution of HC to the formation of nCRF. 

In Fig. 1, the axons of several RCs form synaptic connections with the dendrites of 
a BC. The RCs compose the CRF center of the BC. HC connects many of the nearby 
RCs in the horizontal direction through its dendritic branches. Horizontal cells (HCs) 
also interconnect with each other. The HCs integrate the responses of the RFs, 
transfer them to the BC, and inhibit BC's CRF center, and form BCs CRF surround. 
BCs with antagonistic center- surround CRF transfer their responses into GC and form 
the GCs CRF with antagonistic center-surround structure. AC connects many of the 
nearby GCs in the horizontal direction through its extensive dendritic branches. ACs 
also interconnect with each other. The ACs integrate the responses of the BFs, 
transfer them to the GC, and inhibit BCs CRF surround, thus forming GCs nCRF. 
IPC takes feedback control of HC and BC and changes the size of CRF surround and 
MC takes feedback control of IPC and AC and changes the size of CRF center 
through centrifugal fibers. 
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Fig. 1. Neural circuit for dynamically adjusting Fig. 2. The multi-layer network model for 
RF. processing image. 



3.2 Multi-layer Network Model for Image Processing 

According to the above neural circuit for dynamically adjusting RF, we propose the 
multi-layer network model for processing image shown in Fig. 2. 



3.3 Numerical Calculation Model of GC's RF 

The output of the second layer can be expressed as follow: 

GC(X,Y)= £ JW v RC(x,y)- £ ^W 2 - RC(x,y)+ £ £W 3 • RC(x, y) 



where W* = A, 



U-xnr+lY-jn)' 
of 
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■ As 1 ,W 2 = A 2 -e 



■ As2 .W 3 = A 3 



where, GC(X, Y) is the response of GC; RC(x, y) is image brightness projected onto 
RC within RF; Sj, S 2 and S 3 are CRF center, CRF surround and nCRF respectively; 
W[, W 2 and W 3 are the weighting functions of the RCs within Si, S 2 and S 3 
respectively; Ai, A 2 and A 3 are the peak sensitivities of Si, S 2 and S 3 respectively; Oi, 
o 2 and o 3 are the standard deviations of the three weighting functions respectively, 
and 3o"i, 3g 2 and 3g 3 are the radii of Si, S 2 and S 3 respectively; Asi, As 2 and As 3 are 
the bottom areas of the weights of the RCs in Si, S 2 and S 3 respectively; x and y are 
position coordinates of RC; x and y are center coordinates of RF; X and Y are 
position coordinates of retinal GC. 

In our experiment, pictures are imaged on the central retina with the range of 0-10 
degrees eccentricity. 
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4 Results 

We confirmed the change range of the size of RF of retina GC according to the 
physiological data. So, in the process of experiment, the change range was limited and 
cannot enlarge or reduce arbitrarily. We got the schematic diagram (as shown in 
Fig. 3) of RF dynamic adjustment process after tracking. In Fig. 3, red means that RF 
has reached the final size, and needn't to change; blue means that RF need to enlarge 
further; yellow means that RF need to reduce further; the real line denotes the actual 
size after this iteration; the broken line denotes the RF size that be replaced gradually 
in the process of iteration, which shows the change trail of RF. Observing the final 
size of RF (red RF) in Fig. 3, we found that the smallest RFs are always on the border 
of the image. In the background wall, the information is uniform within each color 
block, so the RFs representing the information within color block reach to the 
maximum. The RFs representing the information of the lamb, pillows, cup, flowerpot, 
the bottom of the sofa don't change when they just reach the edges. Therefore, from 
Fig. 4 we can see that RFs best represent the image information by dynamically 
adjusting their size in the limited change range. 
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Fig. 3. The schematic diagram dynamic Fig. 4. The difference in image 
adjustment process. understanding between human vision and 
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5 Discussion 



The property of nCRF is not changeless. The nCRF can adjust its filter characteristics 
according to the spatial frequency components of images. And it is flexible toward 
changing contrasts and brightness of stimulations. Along with the changing of image 
spatial properties, nCRF sometimes turn into high spatial frequency filter and 
sometimes into low one. The nCRFs of most neurons are facilitatory under the 
condition of low contrast or low brightness, which shows that nCRF enhance the RF 
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response in order to detect contours of large-area image at this moment. After the 
increase of the contrast or the brightness, the nCRF will accordingly weaken the RF 
responses in order to detect the difference of the image inside and outside the RF. In 
large-area integration, this dynamic characteristic of nCRF provides necessary 
condition for visual system detecting Fig. -ground in a variety of stimulus conditions. 
Our current research shows the dynamic characteristic that RF varies with different 
stimuli. 

Now there are two common senses in computer vision, saying that (a) an animal's 
vision system is much cleverer than a machine one, and (b) object recognition needs 
the help from higher level knowledge. To (a), the advantage of a biological vision 
system that is it has a systematic architecture with rich knowledge and rich layers of 
representing and processing. Fig. 4 shows the reason. To (b), what is knowledge 
needs to be clarified. 
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Abstract. This paper addresses the control problem of an underactuated rotary 
crane system by using a recurrent neural network (RNN) and a particle filter (PF) 
based state estimation. The RNN is used as a state feedback controller which is 
designed by a constricted particle swarm optimization (PSO). As the study also 
considers the problem with assuming that the velocities of the system are not 
obtained, PF is utilized to estimate the latent states. Simulations show that the 
RNN could provide a superior evolutionary performance and less computational 
cost compared to a feedforward NN and that the PF is effective in estimating the 
unobserved states. 

Keywords: recurrent neural network, particle swarm optimization, nonlinear 
control, underactuated system, particle filter, sequential Monte Carlo method. 



1 Introduction 

The recent immense growth of computing power has allowed several computation meth- 
ods applying in broad areas. One of the most successful implementations is the use of 
stochastic and intelligent approaches in control engineering [ 1 j . In particular, the ap- 
plications of neural networks (NNs) and evolutionary algorithms (EAs) have brought 
much success and convenience in both offline and online designs [ 1) - 0. 

Aside from several advantages of NNs, such as good nonlinear processing ability 
and robustness with inherently parallel architecture, recurrent NNs (RNNs) have inter- 
esting properties with great potential, such as attractor dynamics and internal memory, 
and they have shown superiority to feedforward NNs (FNNs). Since a recurrent neu- 
ron already has an internal feedback loop, it captures the dynamic response of a system 
without external feedback through tapped delays [4 1 , RNNs are thus dynamic mappings 
and are more appropriate than FNNs when applied to dynamical systems. Nevertheless, 
despite important capabilities, RNNs are much less popular than FNNs because it is 
hard to develop a convenient learning algorithm as well as classic gradient-based al- 
gorithms are apparently insufficient 0. In order to overcome the problems of RNNs 
training, the last two decades have shown an increasing amount of interest in applying 
EAs to construct RNNs (e.g., see 0, 0). Among the EA family, particle swarm opti- 
mization (PSO) [7 1, |8| is known as one of the latest methods and it has been shown to 
have several advantages compared to conventional backpropagation in training FNNs 
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for nonlinear function approximation, such as less computational cost and faster con- 
vergence [9|. In this paper, attention is paid to the use of PSO as a training algorithm to 
construct an RNN-based controller for a nonlinear system. 

On the other hand, due to the fact that in dynamic system control the system state is 
not always observable completely, state estimation is thus an important prerequisite to 
make the operation possible, safe and economical. For example, sensing the velocities 
in mechanical systems is hard or may require more sensors with high cost. We thus 
assume that the velocity states of the system are not observed and hence use a particle 
filter (PF) to estimate the velocities. Known as a stochastic estimation method, PF has 
emerged as a very potential technique during the last decade as it is applicable for 
nonlinear and non-Gaussian systems IflOl . ATI . 

In order to validate the method, this study considers a rotary crane to be the control 
object, which is a reduced two degree-of-freedom model with underactuated behavior. 
Underactuated systems, which are characterized by the fact that they process fewer ac- 
tuators than degrees of freedom, have attracted much research during the last decades 
due to it pervasive applications. Such systems generate interesting but difficult control 
problems as they usually exhibit nonholonomic behavior and complex dynamics. As 
widely used in industry, crane systems have been the subject of several studies, present- 
ing various methods for load swing control including classical and fuzzy approaches 
ITTZI . |Q~3). The crane system being considered herein is hard to stabilize quickly and 
also known to be a nonholonomic system |[T3l . As is well-known, the class of nonholo- 
nomic systems cannot be asymptotically stabilized by continuous time-invariant static 
feedback |14|. Several approaches have been introduced for such systems, presenting 
time varying, discontinuous, dynamic state feedback control methods 1141 . Generally, 
these methods aim to deal with asymptotic stabilization, considering the ideal system 
model while assuming that all system states are observed. This paper addresses the 
control of such a system with supposing that there exist unknown states (velocities). 

2 Nonlinear Control Using a Recurrent Neural Network and 
Particle Swarm Optimization with Particle Filter Estimation 

Considering a discrete, nonlinear, non-Gaussian system represented by: 

xk=fk(xk-i,rik) (1) 

yk = gk(xk,^k) (2) 

where y^ is the vector of observations, Xk is the state vector at time k (k = 0, 1,2...); 
fk(-) is the (known) state transition function, g^(.) is the (known) observation function; 
r\k and ^ are noise vectors. The control task is to regulate the system from an arbitrary 
initial point x lmt to a desired point x* with an assumption that there exist unobserved 
states. To do so, an RNN is used as a state-feedback controller which is optimized by 
PSO and the latent states are estimated by a PF. The schematic of the control system is 
shown in Fig.Q] which also includes a limiter for dealing with the constraints involving 
to the control input (\u\ < w max or \u\ < w max ). 
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Fig. 1. The proposed control system. 



Fig. 2. Recurrent neurocontroller. 



Fig. [2] shows the structure of the RNN used herein. The network consists of three 
layers: the input layer with / neurons, the hidden layer with H neurons, and the output 
layer with O neurons (i.e., it is a / — H — O structured RNN). The input layer contains 
only feedforward connections and there is no recurrent/feedback loop in its own neurons 
or from/to other layers. In contrast, the hidden and output layers are fully recurrent. 

The PSO and PF are employed in two following operators: 

i) Design: Used to design a feedback controller in offline manner. That is, a PSO 
is utilized to evolve the RNN controller. A sum-squared error is used as the evaluation 
function for the PSO, which is: 



£ = ie,-(^ ef -*r d ) 2 +ie m (<f- 



..end 



(3) 



where Qi\ are the weight coefficients, (.) end represents the values at the final state (at 
the end of control simulation), and (.) ref represents the desired values. In most cases, 
such as in this paper, w ref = 0. 

In this study we utilize the constricted version of PSO proposed in (7). After 
finding the local best PB„ = [pb n \, pb„2, •••, pb„o] and the global best gb = 
[gb\, gb2,- ■ ■ ,gbo] in the y'-th iteration, the w-th particle updates its velocity V n = 



[v„i, v n2 , •••, Vbd] and position W„ = [w„i, w n2 , 
(j + 1) with the following equations: 



w„d] in the next iteration 



y'+l 

V nd 



: X {fnd + c i r i (pKj - Kd) + c 2r 2 (gb' d - <,) ) 



(4) 
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where d — 1 , 2, • • • ,D is the dimension number; n = 1 , 2, • • • ,N (N is the swarm popula- 
tion size); j = 1 , 2, • • • , J is the iteration number; c\ and c 2 are positive constants; r\ and 
r% are random values in the range [0, 1] ; and % is the constriction coefficient defined as: 
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1 : for i = 1 : N p do 
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Fig. 3. Particle filtering pseudocode. 



Fig. 4. Model of the rotary crane system. 
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n) Estimation: Used to estimate the latent states of the system. Namely, using the 
incomplete measurements y to obtain the estimate x by PR After the Design finishes 
constructing a controller (using the system model with all sates completely observed), 
the control process of the real plant are then executed while the PF performs in every 
time step (online) to estimate the hidden states. 

In principle, the objective of Bayesian estimation is to estimate the hidden Xk based 
on the set of all available (observed) measurements vi^ = {yt, i — 1,2, ...,k} up to time 
k. To do so, the posterior probability density function (pdf) p(x] c \yi:k) is calculated recur- 
sively. Based on Monte Carlo simulation, PF approximates the posterior by representing 
the required posterior as a finite set of N p random samples (particles) X^ = {#l}i=i an( ^ 
associated weights {(Oi},^. Fig. [3] shows the sampling importance resampling (SIR) 
algorithm, a very commonly used PF. More details of PF approach can be found in 

m, hd. 

3 Rotary Crane System 



The control object to be considered in this work is a rotary crane system as shown in 
Fig. [4] which is also known as a nonholonomic system [13|. Herein, is the rotation 
angle which serves as the control input, r is the radius of rotation, h is the height from the 
tip of the boom, m is the mass of the load, and / is the length of rope which is assumed to 
be not elastic or slack. Assume that the vibration of swinging load is sufficiently small, 
this study only considers the movement of the jib rotating around the Z axis. 
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Suppose that the projection of the load m on X—Y plane is M, its position 
(xm> yti) can be determined by the angle 9. The state of the system is defined as 
x = [x\, X2, jt-3, X4] = [xm, Xm, yM, yti] • The dynamics of the system is: lfT3ll 
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(7) 



where A = yg/l is the natural frequency, g — 9.8[m/s 2 ] is the acceleration of gravity. 

It should be noted here that, in this rotary crane model, the control input is not the 
torque of the crane, but the angle 0. Angle-command system is realizable when using 
a pulse motor [15], the model (7) therefore can be used without the appearance of the 
load mass m. However, it is necessary to consider the limit of the angular velocity of 
the motor in order to avoid the loss of synchronism lfT5l (i.e., \6\ < m ax)- 

The task in the crane system control is to regulate the load mass, by rotating the 
crane jib, from an arbitrary position ;c lnlt (determined by an initial angle 60) to a desired 
position, which can be defined as x rsi = [r, 0, 0, 0] without loss of generality. This 
rotation movement usually accompanies with the vibration or swinging of the load, 
making the control problem more difficult as requiring to suppress the load sway. 

4 Performance Test and Control Simulations 



4.1 Comparative Performance Test 

In order to show that the RNN is a suitable choice, the RNN is compared to a three-layer 
FNN. Under the same instances, the criteria for comparing the two networks are their 
evolutionary performance and computation time. 

i) Evolutionary performance: The evolutionary performance of the two PSO-based 
control systems (using the RNN and FNN) are evaluated by the mean success rate S m , 
i.e., the rate of successfully-evolved controllers over the total performed runs, and the 
mean {global best) error value E m obtained at each iteration. A controller is considered 
to be successfully evolved if it can obtain an error smaller than a required accuracy 
after running the Design by PSO, that is, E < E suc = 10~ 4 . The above two factors are 
calculated over the replications of varying 20 different initial system positions, which 
are random values uniformly distributed in the range Oo <G [— 7T, n], and 50 times of 
randomly changing the initial population of the PSO for each of the initial positions. 

In the PSO, the population size is set to be 20 particles and the number of iterations is 
100. These values are selected to be not too large with respect to computational cost. As 
suggested in Q, the constants in the constricted PSO should be set as c\ = C2 — 2.05, 
hence e = 4.1 and % s=s 0.7298, for assuring the convergence. In our tests, the crane 
system parameters are / = 2[m], r = l[m], the constraint is \6\ < max — 1.0[rad/s], 
and the fourth-order Runge-Kutta method with time step of 0.01 seconds is utilized. 

To demonstrate the advantages of the RNN over FNN, we first tune the number of 
hidden neurons and the range of PSO initial population (based on trial and error) such 
that the FNN achieves the best performance. That is, the number of hidden neurons 
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Fig. 5. Result of comparative evolutionary performance test. 
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Fig. 6. Trajectories of the load mass on X-Y plane. 



in the FNN is H = 7, and the initial population range is [ — 1.5, 1.5]; this range is 
then applied to the RNN. In the RNN, only H = 2 hidden neurons are used, which 
is set to be small with respect to computational cost. Namely, the structures of the 
FNN and RNN are 4 - 7 - 1 and 4-2-1, respectively. It should be noted that the 
above initial population range and the number of hidden neurons used for the RNN are 
not the optimal values for its performance. In both networks, we use a linear function 
/io.act (x) = x as the activation function of the input and output layers and a hyperbolic 
tangent function /h.act W = tanh(x) for the hidden layer. 

The mean success rates and mean global best errors of the PSO in the Design using 
the RNN and FNN are respectively shown in Figs. [5] (a) and (b). It turns out that the 
RNN could obtain a higher mean success rate and lower mean error compared to the 
FNN (the rates at the end of the evolution for the two networks are 97.3% and 91.3%). 

ii) Computational cost: The tests have been implemented in C programming lan- 
guage ran on a PC with a Duo-Core 2.2 GHz CPU and 2 GB of RAM running Ubuntu 
Linux. It was measured that a single run of the Design using the FNN took 1.89 sec- 
onds, while it was only 1.08 seconds in the case of using the RNN. Clearly, the use of 
the RNN noticeably reduced computation time. 



PSO of an RNN Control for a Crane with PF based State Estimation 
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Fig. 7. Control results with 9 = n/2. 



4.2 Control Simulations 

Next, the control simulations are performed with 6q = n/2 and 6q = n , for example. 
Figs. [6] (a) and (b) show the trajectories of the crane using two of the successfully- 
evolved RNN controllers and the PF with 500 particles and threshold = 0.3N P for the 
Estimation, where, for simplicity, it is assumed that there is no disturbance in the system 
but the PF is used only to estimate the hidden velocity states xm and y M based on the 
observed position (xm , Jm)- More details of the control results are plotted in Fig. [7] 
Clearly, the controller is able to regulate the system to the desired position. 

In order to see how well the PF performed in the example, we also plot the errors 
of estimations, i.e., the differences between the real states (velocities) and the estimates 
provided by the PF, which are: 



ejt(t)=xu(t)-XM(t), and e s ,(t) = j>M(t)-y M (t) 



(8) 



It appears that, although there exist errors, the PF could obtain small errors (see 
Fig- 0(d)), revealing that the estimated values are very similar to the real ones. 

5 Discussion and Conclusion 



Although underactuated systems with nonholonomic constraints seem to be a potential 
approach with several advantages bringing it to broad applications, it is a fact that con- 
trol of such systems is a challenging task. The study has presented an intelligent control 
method for an underactuated crane with constraint by using an RNN optimized by PSO 
while considering the problem with hidden velocities which were estimated by PF. 
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Tests were carried out for comparing the RNN and FNN based controllers with small 
values of iteration number and population size. As it is our priority to achieve a good 
evolutionary performance under a given condition, the parameters of the FNN (i.e., 
the number of hidden neurons and the range of its initial connection weights) were 
thus tuned for the most suitable values. Although the parameters of the RNN were 
set unfairly in the comparison, the RNN has provided an attractive performance while 
significantly reduced the computation time. Aside from the self-connected neurons, the 
RNN used in this paper includes the hidden and output layers that have other feedbacks 
interacting each other. The RNN is thus considered to be suitable for such a sensitive 
nonlinear system like the crane. 
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Abstract. Although contemporary information systems are intensively utilized 
in enterprises, their actual impact in automating complex business process is 
still constrained by the difficulties coincided in design phase. In this study, a 
hybrid data analysis methodology to business process modeling that is based on 
using from-to chart is further enhanced to discover connection types and 
include in the process model. From-to chart is basically used as the front-end to 
figure out the observed transitions among the activities in event logs. The 
derived raw patterns are converted into activity sequence on from-to chart by 
using Genetic Algorithms. Finally a process model including AND/OR 
connection types is constructed on the basis of this activity sequence. 

Keywords: From-to Chart, Business Process Modeling (BPM), Process 
Mining, Connection Types, Genetic Algorithms (GA), Event Logs. 



1 Introduction 

In conventional process modeling, a reference model for the target process is 
constructed. However, sometimes the actual enactment may deviate from the proposed 
reference model due to the reasons such as subjectivity of the domain experts or 
incomplete design phase. Process mining is proposed a remedy to these discrepancies 
by mining interesting patterns from the event logs and discovering the business process 
model automatically [5,6,7,8,9,10]. Unlike contemporary process modeling approaches, 
process mining is not biased or restrictive by normative perceptions [14]. 

A new approach for process mining based on using from-to chart is given in [2]. In 
this approach, from-to chart, which is fundamentally used in tracing material handling 
routes on the production floor [4], is adapted to the business process modeling (BPM) 
domain and is used for analyzing the event logs. The transitions among activities 
occurred in process instances are kept and analyzed in a from-to chart in order to 
check if there exists any specific order of the occurrences for representing in process 
model [2]. This approach is further improved in [3] in such a way that the runtime 
complexity of the approach is improved by adopting Genetic Algorithms (GA) [11]. 

In this work, we extend the work in [3] by extraction of AND/OR connection types. 
Through this extension, the type of connections are discovered for each parallel 
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predecessor or successor of underlying activities by interpreting (i) the structure of 
dependency/frequency graph and (ii) the final scores at from-to chart. Then 
dependency/frequency graph is converted into a block-oriented model named control 
flow graph. By this way, the process model is enhanced from a sequence form to a 
more extended form including connections. 

This paper is organized as follows: Section 2 introduces the proposed approach. In 
Section 3, the application of the presented approach on a real case is described. 
Section 4 includes the related work. Finally, Section 5 presents the concluding 
remarks. 

2 Extracting Connection Types in From-to Chart Based Process 
Discovery 

The proposed approach consists of three basic steps: construction and rearrangement 
of from-to chart, relation extraction and constructing of process model that includes 
connection types. 

2.1 Construction and Rearrangement of From-to Chart 

The first operation of the proposed methodology is the creation of a fromtochart 
database table by retrieving the activity types from the event logs and populating 
fromtochart table. Predecessor and successor for each transition are parsed in every 
process cycle, which is ordered by timestamp in ascending order and the current score 
at (predecessor,successor)' /l element at fromtochart table is incremented. 

Then scores at fromtochart table are evaluated in order to eliminate the effect of 
weak scores on finding the fittest activity sequence by pruning down them prior to 
rearrangement. Basically there are three evaluation metrics used in the proposed 
methodology: confidence for from-to chart (confidence FTC), support for from-to 
chart (support FTC) and modified lift [2]. 

Rearrangement operation is the engine component of proposed methodology, 
which aims to find out the activity sequence with the minimum total moment value at 
fromtochart table in GA fashion. The coarse-grained GA stages are detailed in [3]. 

2.2 Relation Extraction 

Basically there are three types of relation represented in dependency/frequency graph: 

a. Immediate Succession. Let S be an optimum activity sequence over T, a range of 
activities, i.e. S e T (i.e. a permutation of 7) and A, B e T. Then B immediately 
succeeds A, if and only if position of A (p A ) and position of B (p B ) at S are two 
successive integers such that; p B = p A + 1, where p A and p B e [1, \S\]. 

b. Succession. Let S be an optimum activity sequence over T, a range of activities, 
i.e. S e T and A, B e T. Then B succeeds A, if and only if position of A (p A ) and 
position of B (p B ) at S are two integers such that; p B > p A , where p A and p B g [1, 
\S\]. 
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c. Back-tracking. Let S be an optimum activity sequence over T, a range of 
activities, i.e. S e T and A, B e T. Then B backtracks A if and only if position of 
A (p A ) and position of B (p B ) at S are two integers such that; p B < p A , where p A and 

Pb e [1, 1517. 

2.3 Process Model Construction by Deriving AND/OR Connections 

AND/OR connection type extraction handles relations of each activity type, which are 
held in successor and predecessor activity list, in a pair-wise fashion. There are two 
formulations for calculation connection scores with respect to the type of connections 
(i.e. join or split) as stated in [13]: 



■BaC, 



B>C\ + \C>B\ 
\B>A\ + \C>A\ 



•BaC, 



|fi>C| + |C>B| 

\a>b\ + \a>c\ 



(1) 



While A => B A C S plit term indicates the score of potential split-type connection 
between activities B and C, which are successors of activity A, A => B A C JO in term 
indicates the score of potential join-type connection between activities B and C, 
which are predecessors of activity A in dependency/frequency graph. 

The denominators of these formulations, IA > Bl + IA > CI and IB > Al + IC > Al, 
indicate the total number of outgoing or incoming observations. The numerator, IB > 
CI + IC > Bl, is the number of times that activity B and C appear directly after each 
other. Quantitatively connection scores, which are greater than the AND threshold, 
implies an AND-connection between the related activities. Otherwise it is an 
OR-connection. 

3 Application of the Proposed Approach 

The sample business process, which we use for illustrating the approach, consists of 
ten activity types: intern repair, make ticket, arrange survey, inform client survey, 
survey, repair ready, send ticket to financial admin, ready inform client, ticket ready 
and first contact. Table 1 shows the initial state of fromtochart table for 333 process 
instances. Note that f rcon activity is denoted as the initiator activity. 

Table 1. Initial and Final States of fromtochart Table (Element a 6 6 is set to zero due to 
evalution operation). 
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After fromtochart table is created and parsed transitions in activity streams are 
populated to this table, scores are evaluated with respect to support and confidence 
thresholds (supportFTC=0 .40 and confidenceFTC=0 .40). Table 1 shows the initial 
and final states of fromtochart table. 

After scores are evaluated, GA engine reproduces activity sequence starting with 
fircon activity and then calculates fitness score for these activity sequences 
according to proposed population size (popuiationsize=8 0) and maximum number 
of iterations (maxiterationNum=200) settings. 

As shown in Figure 1, initial population is quite diverse early on the process, so 
crossover frequently takes large steps early in the search space and fitness scores of 
the individuals tend to increase. Afterwards smaller steps are taken when most 
individuals are quite similar, i.e. the gap between average and maximum fitness score 
boundaries has shrunk. Consequently the fittest (and optimum) activity sequence is 

obtained as {firCon, makTic, arrSur, infCliSur, sur, intRep, repRdy, rdylnfcli, 

senTic, ticRdy} with a 0.779 fitness score. 



fitness score vs. Iteration 
[MifMjlrt1iufi\i,'F -.iKl, rririx. iterril nTh): ^M) 



-~BKISSS8fi;£KK!BSS!8BgK*SSSia|e88&j53ag|jjgg! 
Iteration 



(. fitness "aya. fitness ■ mil. fitness 



Fig. 1. Runtime Analysis for Rearrangement Operation. 

After rearrangement of fromtochart table, relations are extracted according to 
positive scores remarked at fromtochart table. Consequently, dependency/frequency 
graph, which visually converts the fittest activity sequence (genotype) into the process 
model (phenotype), is constructed as shown in Figure 2. 



Fig. 2. Extracted Relations in Dependency/Frequency Graph. 
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As the final operation, type of connections at successor and predecessor activity list of 
each activity are derived with respect to the AND threshold (ANDthreshoid=o.io). 
Hence dependency/frequency graph is converted into control flow graph as shown in 
Figure 3. Sample connection score calculations are as follows; 



sur => intRep a repRdy s/ , l 



158 



158 + 154 



= 0.505, then AND — split type connection 



repRdy => senTic a rdylnfCli sfut ~ — 1, then AND — split type connection 



rdylnfCli => repRdy a senTic KnN 
senTic => rdylnfCli a ticRdy SPLIT = 
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= 0.438, then AND — join type connection 
0.438, then AND — split type connection 



















... 




\ 








V 










\ 












^ 




; 






Fig. 3. Derived Connections in Control Flow Graph. 

For this data set, the percentage of transitions in testing dataset that are represented in 
the dependency/frequency graph (i.e. recall) is 99.85%. Additionally, all extracted 
relations have corresponding complements in testing dataset (i.e. precision of the 
discovered process model is 100%) and a 58.87% improvement is achieved in 
processing time by the use of GA based rearrangement. 

4 Related Work 



In [5], Aalst et al. highlights the process mining usage in analysis and improvement of 
process flexibility. It proposes the enrichment of obtained event logs with semantic 
information. Likewise [5], [6] aims to handle both case and control-flow perspectives 
in an integrated fashion. While production workflow is strongly process-oriented, the 
case handling paradigm basically concentrates on the case itself, which is the 
fundamental object to be produced. 

The idea of applying process mining in the context of workflow management was 
first introduced in [7]. In this study, two issues are handled. The first issue is to 
discover a workflow graph generating activities appearing in a given workflow log. 
The second issue is to find the definitions of the relation conditions. 

Cook and Wolf have investigated similar issues in the context of software 
engineering process. In [8], they describe three methodologies for process discovery 
ranging from the purely algorithmic to purely statistical: RNET, KTAIL and 
MARKOV. 
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In [9], Weijters and Aalst propose a rediscovery technique to deal with noise and to 
validate workflow processes by uncovering and measuring the discrepancies between 
the prescriptive models and actual process executions. Detecting concurrency appears 
as one of the fundamental concerns. Therefore AND/OR connectors are aimed to be 
explicitly distinguished in the process model. Moreover, local and global metrics are 
proposed to find explicit representations for a broad range of process models. These 
metrics are revised in [1]. In [10], the proposed method, named alpha algorithm, seeks 
to rediscover sound Workflow nets. Given such an algorithm, it is aimed to find the 
class of workflow nets which can be rediscovered. As a way of representation, it 
generates concrete Petri nets for a broad range of process models rather than a set of 
dependency relation between events like in [9]. 

As a GA implementation in facility layout problem (FLP), Sarker et al. attempts to 
handle locating multiple identical machines in a linear layout, which is called tertiary 
assignment problem (TAP) in [12]. It is attempted to solve this problem type by 
identifying sets of identical machines which may be partitioned into individual, 
unique machines. In order to relax the underlying FLP and provide approximate 
solutions, amoebic matrix is used. 



5 Conclusion 

Process design is a complicated, time-consuming phase and typically, there may arise 
incompatibilities between process design and process enactment. Process mining 
techniques aim to handle such incompatibilities by extracting information on the 
process from the event logs. In this work, we present an enhancement on a hybrid 
process mining approach that uses from-to chart, in order to extract connection types 
and to form a process model. From-to chart, which is a fundamental analytical tool 
used in tracing material handling routes at production floor, is adapted to process 
modeling domain in order to analyze transitions among activities. The activity 
sequence, which is rearranged in a straight-line form by developed GA engine, 
constitutes the backbone of dependency/frequency graph without compromising 
accuracy. In order to construct the process model, the generated 
dependency/frequency graph is converted into control flow graph according to AND 
threshold. The applicability of the approach is demonstrated on a case study. 
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Abstract. A classification problem is fully partitioned into several small 
problenrs each of which is responsible for solving a fraction of the origi- 
nal problem. In this paper, a new approach using class-based partition- 
ing is proposed to improve the performance of genetic-based classifiers. 
Rules are defined with fuzzy genes to represent variable length rules. 
We experimentally evaluate our approach on four different data sets 
and demonstrate that our algorithm can improve classification rate com- 
pared to normal Rule-based classification GAs [I] with non-partitioned 
techniques. 

Keywords: Genetic Algorithm, Rule-based Classification, Class 
Decomposition, Divide and Conquer, Fuzzy Rules. 



1 Introduction 

The interest in decision-making has been gaining momentum in recent years. 
Nowadays enormous amounts of information are collected continuously. The 
tremendously growing amount of data has made manual analysis by experts 
a tedious task and sometimes impossible. Many hidden and potentially useful 
relationships may not be recognized by the analyst. The explosive growth of data 
requires an automated way to extract useful knowledge. One of the possible ap- 
proaches to this problem is by means of data mining or knowledge discovery 
from databases. Rule-based classification |2l3j is one of the most studied tasks in 
data mining community and is an active research area. Classification rules are 
typically useful in medical problems, specially in medical diagnosis. Such rules 
can be verified by medical experts and may provide better understanding of the 
problem by-hand. 

Numerous techniques have been applied to classification in data mining over 
the past few decades, such as statistical methods, expert systems, artificial neural 
networks, database systems, and evolutionary algorithms ( [3], [5], [B], [7], [5]). 
Among them, genetic algorithm (GA)-based approaches have attracted much 
attention and become one of the popular techniques for classification 9 . De 
Jong and Spears [T] considered the application of GA to a symbolic learning 
task — supervised concept learning from a set of examples. Corcoran and Sen 
[2] used GA to evolve a set of classification rules with real-valued attributes. 
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Ishibuchi et al. [3] examined the performance of a fuzzy genetic-based machine 
learning method for pattern classification problems with continuous attributes. 

However, when GA is applied to larger-scale real- world classification problems, 
it still suffers from some drawbacks such as the inefficiency in searching large 
solution spaces, the difficulty in handling the internal interference of training 
data, and the possibility of getting trapped in local optima. A natural approach 
to overcome these drawbacks is to decompose the original task into several sub- 
tasks based on certain techniques. The purpose of decomposition methodology 
is to break down a complex problem into several manageable sub-problems while 
providing attention to each subproblem independently. 

According to Michie [TO], finding a good decomposition is a major tactic, 
both for ensuring the transparent solutions and for avoiding the combinato- 
rial explosion. It is generally believed that problem decomposition can bene- 
fit from conceptual simplification of the problem, making the problem more 
feasible by reducing its dimensionality, achieving clearer results (more under- 
standable), reducing run time by solving smaller problems and using parallel 
or distributed computation, and allowing different solution techniques for indi- 
vidual sub-problems. Various task decomposition methods have been proposed. 
These methods can be roughly classified into the following categories: functional 
modularity, domain modularity, class decomposition, and state decomposition, 
according to different partition strategies [11113] . 

However, most of them are used in artifcial neural networks (ANN), very few 
find their applications in GA, especially GA-based classification [T3]. As GAs 
have been widely used as basic soft computing techniques, the exploration of 
class decomposition with GAs becomes more important. Decomposition methods 
have been used in various fields, such as classification, data mining, clustering, 
etc. [15 presented a feature decomposition approach for improving supervised 
learning tasks. The original set of features is decomposed into several subsets. 
A classification model is built for each subset, and then all generated models 
are combined. A greedy procedure is developed to decompose the input features 
set into subsets and to build a classification model for each subset separately. 
Wcile and Michielssen [TO] explored the application of domain decomposition 
GAs to the design of frequency selective surfaces. Masulli and Valentini [T7] pre- 
sented a new machine learning model for classification problems. It decomposes 
multi-class classification problems into sets of two-class sub-problems which are 
assigned to non-linear dichotomizers. Chan and Zhsu [15] proposed a similar 
approach using GA and used Fisher's linear discriminant (FLD) Algorithm to 
re-assemble the rules. Apte et al. |19j presented a new measure to determine the 
degree of dissimilarity between two given problems and suggested a way to search 
for a strategic splitting of the feature space that identifies different characteris- 
tics. Watson and Pollack |20j used techniques from multi-objective optimization 
to devise an automatic problem decomposition algorithm that solves test prob- 
lems effectively. 

In this work we propose a new solution based on the class decomposition 
approach, in which a classification problem is fully partitioned into N class 



Class Decomposition for Building GA with Fuzzy Rule-Based Classifiers 69 

modules; where each module is responsible for developing a classifier for a specific 
class. In other words, each class receives enough attention whatever the size of 
the input data that represents that class. Accordingly, rare occurring classes 
will not be ignored and still get their own attention. These modules are trained 
in parallel and independently and the results obtained are integrated to a final 
solution. Several benefits are gained using this class decomposition technique: 
all classes are fairly represented; parallelism can be obtained for faster solution; 
and scalability is possible as the problem is broken into several manageable sub- 
problems. After this detailed introduction and related work review given in this 
section, details about the methodology of class decomposition using a GA is 
given in section [51 In section four experiments are conducted to evaluate the 
proposed technique. Finally, a brief conclusion is given in sectional 

2 Methodology 

A GA is a search and optimization methodology from the field of evolutionary 
computation that was invented by Holland [3T] . A GA is based on the Darwin's 
natural selection principle of the survival of the fittest, and is widely used for 
hard problems in engineering and computer science. A GA is a population-based 
method where each individual of the population represents a candidate solution 
for the target problem. This population of solutions is evolved throughout sev- 
eral generations, starting from a randomly generated one, in general. During 
each generation of the evolutionary process, each individual of the population 
is evaluated by a fitness function, which measures how good the solution rep- 
resented by the individual is for the target problem. From a given generation 
to another, some parent individuals (usually those having the highest fitness) 
produce "offspring", i.e., new individuals that inherit some features from their 
parents, whereas others (with low fitness) are discarded, following Darwin's prin- 
ciple of natural selection. The selection of the parents is based on a probabilistic 
process, biased by their fitness value. We use rank-selection mechanism which 
means that a ranking process is performed on the fitness values then individuals 
with the higher fitness value are selected for the next generation. The generation 
of new offspring from the selected parents of the current generation is accom- 
plished by means of genetic operators. This process is iteratively repeated until 
a stop criteria is reached. 

2.1 Individuals Representation 

An individual in our method is a classification rule where each gene represents 
the minimum and maximum values of intervals of each attribute that belongs to 
such rule. In rule-based classification, there are various representation methods 
in terms of rule properties (fuzzy or non- fuzzy) and attribute properties (nominal 
or continuous). In our approach, we use fuzzy IF-THEN rules with continuous 
attributes. A rule set (classifier) consisting of a - user determined - number of 
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rules as a solution candidate for a classification problem. An IF-THEN rule is 
represented as follows: 

Ri : IF (Vlmin < X\ < Vlmax) A (Vlmin < X 2 < V 2m ax) A A (Vnmin < X n < 

V„max) Then y = C 

where Ri is a rule and each rule represent a chromosome in which each gene 
represents an attribute and the consequence gene stands for a class; n is the 
number of attributes with (x%,X2, ••■X n ) are the input attribute set; and y is 
the output class category assigned with a value of C. Vjmin and Vj max are the 
minimum and maximum bounds of the jth feature Xj respectively. We encode 
rule Ri according to Figur^T] 



Genel(Ai) Genen(A„ 

1 vlmin Vlmax ^n vnmin Vn 

Fig. 1. An individual representation. 



where the weight Wj is a real- valued variable taking values in the range [0..1]. 
This variable indicates whether or not the potential attribute is present in the 
corresponding classification rule. More precisely, when Wj is smaller than a user- 
defined threshold (called Limit) the attribute will not appear in the related rule. 
Therefore, the greater the value of the threshold Limit, the smaller the probabil- 
ity that the corresponding attribute will be included in the rule. Variable length 
rules is implemented by using weights, which simplifies the problem by ignoring 
insignificant attributes randomly. The weights are fuzzy indicators representing 
attribute significance. The less their significance than a user defined threshold, 
the attribute is then ignored. Vj m i n and Vj ma x are the limits of the intervals 
corresponding to the attribute Ai . Note that the above encoding is quite flexible 
with respect to the length of the rules. A traditional GA is very limited in this 
aspect, since it can only cope with fixed-length rule. In our approach, although 
each individual has a fixed length, the genes are interpreted (based on the value 
of the weight Wi ) in such a way that the individual phenotype (the rule) has 
a variable length. The start of the first population consists of generating, arbi- 
trarily, a fixed number of individuals during the evolution. 



2.2 Genetic Operators 

For the developed method, the usual one-point crossover operator is stochas- 
tically applied with a predefined probability, using two individuals of the se- 
lected pool. The crossover point is a percentage of the length of the individual 
that defines the starting point from where the crossover breaks the string. We 
use arithmetic crossover method in the experiments |22j . The employed method 
works as follows: 
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Consider two chromosomes C\ = (c\, , c*) and C2 = (c\, , c n)- Ap- 

plying the crossover operator on C\ and C2 generates two offspring H\ = 
(c 1; .., q , c i+1 , ...., c„) and i^2 = (c l7 .., c i , c i+1 , ...., c„). 

The mutation operator is used to foster more exploration of the search space 
and to avoid unrecoverable loss of genetic material that leads to premature con- 
vergence to some local minima. In general, mutation is implemented by changing 
the value of a specific position of an individual with a given probability, denom- 
inated mutation probability. We developed new mutation operators tailored for 
our genome representation that Shift the starting location and/or the ending 
location towards the right or the left: the value in the starting/ending location 
of a randomly selected gene is increased or decreased by a fuzzy fraction of the 
upper and lower limits. This is implemented in such a way that the lower and 
upper bounds of the domain of the field are never exceeded. 

2.3 Fitness Function 

As each chromosome in our approach comprises one rule, the fitness function 
actually measures the collective behavior of the rule. 

Definition 1. Accuracy of a certain rule is the ratio of correctly classified in- 
stances by this rule to the whole number of instance to which this rule satisfied 

C Number of correctly classified instances by this rule 
n Total number of instances satisfy this rule 

2.4 Ending Criteria 

As the GA searches the solution space, defined by the representation of the 
application at hand, some of the rules are selected in the second step as promising 
candidate solutions based on their evaluated fitness. They will undergo genetic 
operations to generate a new population. This process is repeated until the end 
criteria is met; either that the algorithm has reached a good enough solution or 
that the maximum number of generations have been iterated. We use a hybrid 
ending criteria that exits the execution whenever a good solution is met (human 
expert defined) otherwise the execution will continue after maximum number of 
generations. 

2.5 Class Decomposition 

Classification techniques usually assume homogeneity of class representation (the 
number of cases represent each class is close) ; therefore the absence of a class in 
the result classifer due to the trivial existence of that class in the training data is 
possible. Our technique addresses this problem by building a separate classifier 
for each class and then integrate them into one classifer. We used the training 
data of all classes to calculate the accuracy of each classifer (for each class) to 
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minimize the connecting rules. Let us assume a classification problem that has 
c classes in the n-dimensional attribute space. The task of classification is to 
assign instances to one out of the pre-defined c classes, by discovering certain 
relation among the attributes. Then, the discovered rules can be evaluated by 
classification accuracy or error rate either on the training data or test data. 
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Fig. 2. Class Decomposition. 



A traditional GA maps attributes to classes directly in a batch manner, which 
means all the attributes, classes, and training data are used together to train a 
group of GA chromosomes. In our approach, a GA is executed with class decom- 
position. As shown in Figure [H the proposed solution consists of three steps. 
First, the original problem is divided into c sub-problems in terms of classes. 
Then, c GA modules are constructed for these sub-problems, where each mod- 
ule will be responsible for evolving a sub-solution. Finally, these sub-solutions 
are integrated to form the final solution of the original problem. 

Definition 2. Consider having c classes in the classification problem on hand, 
with n-dimensional pattern space, and p vectors where Xi — {xi\, Xii, . . . , Xi n ), 
i = 1,2, ... ,p,p <C c are input instances. Accordingly, the given classification 
problem can be denoted as follows: 



f:X-+T 



where X £ R n is the set of instances with n attributes, and T £ R c is the set of 
output classes. Required a mapping f that maximizes accuracy. Assume that the 
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c- class problem is divided into c subproblems, each contains the vectors Xi that 
present T J . Hence the class set can be denoted as 

r = T i(j T 2(j ...y T c 

each subproblem can be formulated as required fj with an optimal classification 
accuracy and score on T J 

fj:X->T*. 

Having c subproblems, c GA sub-modules are constructed and could be executed 
in parallel. 

2.6 Integration 

Although each GA module has evolved a portion of the solution, we cannot just 
simply aggregate their sub-solutions as the final one, because each GA module 
only classifies only one class. Therefore, when the sub-solutions are combined 
together, there may still exist conflicts among the sub-solutions. For example, 
rules from different modules may classify an instance into several classes. In 
order to resolve these conflicts and further improve the classification rate, the 
classifier employs some intelligent decision rules. The detailed integration process 
is explained as follows. 

The classifier constructs an overall rule set by aggregating all rules from c 
modules. Some decision rules are added to help resolve the above-mentioned 
conflicts. We believe that the ending classification rate obtained from each mod- 
ule would be useful for this purpose. Currently, the following decision rules have 
been employed: 

i) If an instance is classified into more than one class categories by the rule 
set, it will be classified into the one whose corresponding module achieves the 
highest classification rate in the training phase. 

ii) If an instance is not classified into any class category by the rule set, it 
will be classified into the class whose corresponding module achieves the lowest 
classification rate in the training phase, if available. 

2.7 Proposed Algorithm 

To sum up, the process employed in this paper can be summarized by the fol- 
lowing algorithm: 

Input: Population size N; Maximum number of generations G; Crossover prob- 
ability p c ; Mutation rate p m . 

Output: Classifier 

1. GP = 4> 

2. for each Class C, do 

(a) F i :=Initialize(P,C i ) 

(b) while the termination criterion is not satisfied do 
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i. P/:= Genetic Opcrators(Pi) 
ii. Pi'.— Rank and select fittest (Pj U P/) 

(c) end while 

(d) GP = GP U P 

3. end for each 

4. return (GP) 

First, an initial Global Population GP is initialized and for each class Cj, an 
initial population Pi is generated in Step 2. a. Crossover and mutation operations 
are applied to each pair in P$ to generate the offspring population P/ in Step 
2.b.i. The next population is constructed by choosing good solutions from the 
merged population Pj U P/. A ranking based on fitness function is performed 
to select the fittest rules from the new generation to act as basis for the next 
generation, steps 2.b.i and 2.b.ii are repeated to obtain the set of rules (classifier) 
of the current class, in step 2.d the rule set P* is added to the GP. Step 2 is to 
be repeated for each class, finally GP should contain all rules needed to identify 
all classes. 



3 Experimental Results 

We have implemented several classifiers running on four benchmark data sets to 
evaluate our approaches. The data sets chosen are the wine data, glass data, iris 
data and breast cancer data. They are all available in the UCI machine learning 
repository [23]. They all are real- world problems. We partition each data set 
into two parts with an equal number of instances. One half is for training, and 
the other half is for testing. We use the training data to train the rule set, 
and test the generalization power of resulting rule set with the test data. All 
experiments are completed on Pentium 1.8 GHz Dual Core, 1GB RAM PC. The 
results reported are averaged over five independent runs. The parameters, such 
as mutation rate used was 0.07, crossover rate was 0.7, generation limit used is 
3000, and population size was 200. 

We applied the four data sets to our proposed technique and also compared 
them by running them on Kaya's GA [Mj and a normal GA where all classes 
are given as input without decomposition. 

The proposed technique produced classification rules with higher accuracy 
as can be seen in table [1] Besides achieving higher accuracy, performance was 
increased through the parallel execution of the different modules of the problem. 



Table 1. Comparison of the classifiers performance on the four test datasets. 





Normal GA 


Kaya's 


Proposed Tech. 


Wine 


0.29 


0.81 


0.89 


Glass 


0.15 


0.51 


0.78 


Iris 


0.89 


0.93 


0.90 


Cancer 


0.33 


0.47 


0.64 
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4 Conclusion 

This paper proposed a new evolutionary class decomposition approach for clas- 
sification tasks. A classification problem is decomposed into several modules and 
each module is responsible for solving a fraction of the original problem. These 
modules are trained in parallel and the sub-solutions obtained from them are 
integrated to further obtain a final solution. To evaluate our method, we have 
conducted some experiments. The results have shown that our algorithm is ef- 
fective, efficient and promising. 
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Abstract. Constructive multi-start search algorithms are commonly used to 
address combinatorial optimization problems. Multi-start algorithms recover 
from local optima by restarting, which can lead to redundant configurations 
when search paths converge. In this paper, we investigate ways to minimize 
redundancy using record keeping and analyze several restart algorithms in the 
context of iterative hill climbing with applications to the traveling salesman 
problem. Experimental results identify the best performing restart algorithms. 

Keywords: Combinatorial optimization, traveling salesman problem, iterative 
hill climbing, multi-start algorithms. 



1 Introduction 

Intuitively, combinatorial optimization problems (COPs) compute the "most 
favorable" outcome to a problem from a set of possible outcomes. A most favorable 
outcome may be the outcome with the highest profit, lowest cost, shortest distance, 
etc., depending on the problem being solved. For example, shipping companies have 
used COPs to find delivery routes that minimize left turns; by eliminating left turns, 
drivers spend less time in traffic waiting to turn, and use less fuel, resulting in 
monetary savings for the company. Clearly, COPs have great practical importance. 

Many COPs are NP complete and characterized by exponential growth [1-3]; i.e., 
as the size of problem parameters grows, the number of potential solutions grows 
exponentially, causing an exhaustive search of outcomes to be impractical. 
A heuristic search technique is often used in order to assign weights to outcomes, or 
partial outcomes, such that partial solutions that appear to be leading to poor solutions 
are not considered, thus, drastically reducing the search space of outcomes. 
A drawback to heuristic techniques is that the optimal solution may not be found, but 
often a very good solution can be guaranteed. Therefore, heuristic techniques are 
applicable only when a sub-optimal solution is acceptable, or when the faster 
computation times of heuristic algorithms for COPs are required. 

It is well known that many heuristic algorithms explore the same partial solution 
(i.e., a state in the state space) more than one time, thereby wasting computational 
resources. Several researchers have proposed record keeping mechanisms as a 
solution to this problem [4, 5], which we further investigate in this paper. In this 
sense, record keeping resembles Tabu search where the Tabu list includes a subset of 
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the states encountered so far [6, 7]. It differs from Tabu in that the objective of short 
term Tabu memory (e.g., a cache) is to explore the solution space more intelligently 
rather than to speed the search [7] . The goal of long-term Tabu is to enable revisiting 
explored search states in order to attempt to find a better local optimum [6, 7]. 

In [4], we analyzed the performance of a cache as a means for record keeping. In 
[8], we investigated several other record keeping mechanisms such as dedicated 
memory, and Bloom filters [9]. In this paper, we investigate the performance of 
record keeping multi-start COP algorithms using various restart algorithms where the 
record keeping mechanism is a cache. We investigate these in the context of the 
travelling salesman problem (TSP), using iterative hill climbing (IHC) heuristic 
search algorithm as the multi-start heuristic procedure [2, 3, 10]. The TSP is chosen 
since it is a classic, well-understood COP with many practical applications [10]. 

In this paper, we investigate the performance of six construction (restart) 
algorithms used in the context of IHC with randomly generated and benchmark TSP 
problems. The performance of the restart algorithms is empirically determined and 
the algorithms are compared for performance and solution quality. 

The second contribution of this paper is to determine the amount of redundancy 
associated with various construction algorithms, and utilize record keeping techniques 
to reduce duplicate path exploration. We show empirically that a good choice of 
construction algorithm and record keeping mechanism provides improved 
convergence time. The evaluation of construction algorithms and the use of record 
keeping have been studied in previous work [4, 8]. Our work differs from that work in 
that we consider a larger group of construction algorithms, we provide a more in- 
depth study of the effects of different cache configurations, and we provide a 
comparison of cache performance to unbounded memory record keeping. 

2 Construction Algorithms 

In the case of the TSP, IHC can be implemented as a constructive multi-start search, 
in which multiple solutions are computed by constructing solutions from an initial 
solution. At each step, the concept of a neighborhood is used in which a 
neighborhood of a solution s is defined as the set of solutions S that are generated by 
making a minor modification, denoted a move, to s [11, 12]. For example, a TSP tour 
can be adjusted by an operation such as 2-opt which results in exchanging the 
visitation order of two cities [2]. The IHC algorithm proceeds by choosing the best 
s' E S as the next step. The move s is then assigned s' and the process is repeated 
until no improvements to s can be made. This solution is referred to as the locally 
optimal solution. In general, the algorithm repeatedly restarts with a new starting 
configuration until a sufficiently good solution is reached, or a set running time has 
expired. The goal of this paper is to determine the performance of various restart 
algorithms in generating solutions to the TSP in the context of iterative hill climbing 
and record keeping. The restart algorithms evaluated, described in details in ref. [8], 
axe: I) Greedy Enumeration, 2) Greedy Jump, 3) GRASP [12], 4) Nearest Neighbor 
[2], 5) Clarke-Wright [2], and 6) Random. Identical neighborhood generating 
algorithms are used for all implementations. 
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2.1 Record Keeping 

A problem with the IHC using a constructive multi-start search method is that the 
same solution may be reached from multiple starting configurations. Therefore, many 
tours chosen are duplicate tours that have been explored in previous iterations of the 
algorithm. One method to combat this problem is to utilize record keeping 
mechanisms that record some or all tours that have been previously computed. Thus, 
if a tour is generated that has been considered previously, the algorithm restarts with a 
new starting configuration. We investigate two models of record keeping [4, 8]: 1) 
Unbounded Memory, 2) Software Emulation of a Cache [13]. We use set associative 
caches with small set sizes in our experiments. 

3 Experiments 

All experiments were performed using a set of TSP graphs from TSPLIB [14] and 
random graphs with a maximum of 100 vertices. The set of graphs consists of: 

• 10 randomly-generated Euclidean graphs, 100 vertices each 

• 10 random non-planar graphs, 100 vertices each 

• 18 benchmark instances from TSPLIB [14], vertex counts of 16-100 

• 400 random Euclidean graphs of 100 vertices each. 

3.1 TSP Solution Quality 

The experiment results are summarized in Table 1. Each row in the table shows the 
average quality of the solutions found by each of the algorithms in each of the 
experiments as a percentage of the best tour found by Concorde; lower numbers 
indicate better quality, with 100.0 being the best tour discovered. Note that the best 
tour may be found by multiple construction algorithms. 

In the first set of experiments the IHC algorithm is executed using 10 randomly 
generated non-planar graphs. The average performance of the algorithms relative to 
the Concorde results, which are known to be Optimal with respect to this set of 
graphs, is shown in the first row of Table 1. Lower numbers are better with an ideal 
being 100 The second experiment runs the IHC algorithm using each of the 
construction algorithms over ten randomly generated planar Euclidean graphs are 
summarized in the second row of Table 1 . The third experiment uses the TSPLIB 
benchmarks as the input for the construction algorithms [14]. 

The results of the experiments show that the nearest neighbor algorithm performs 
the worst of the six construction algorithms, and that the greedy based approaches 
perform well regardless of the structure of the input. Similarly, the Clarke- Wright 

Table 1. Average performance of the construction algorithms. 
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Random 
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119.08 
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Random Euclidean graphs 
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100.83 


100.50 


100.29 


102.80 


110.43 


TSPLIB benchmarks 


100.66 


100.37 
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100.18 
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algorithm performs relatively poorly regardless the structure of the input graph. The 
random algorithm performed the best of the six algorithms on two of the data sets, but 
its poor performance on the non-planar graph data set brings its average performance 
to be less than the greedy approaches, and indicates that it is sensitive to the structure 
of the input graph. 

3.2 Construction Algorithms and Tour Redundancy 

The next set of experiments examines the quality and redundancy associated with some of 
the construction algorithms. Because the greedy algorithms perform among the best 
despite the structure of input graphs and due to the fact that the random restart does not 
exploit the record keeping efficiently, we perform this set of experiment only on them. 
Figure la shows the average redundancy and tour qualities produced when running IHC 
with the greedy construction algorithms. The average quality of all algorithms is nearly 
identical, but the greedy-jump algorithm has far less redundancy than the others. Figures 
lb, lc, and Id show the distribution of solution quality for 400 random graphs solved 
using Concorde, GRASP, and GE. All the graphs show a distribution that is close to 
normal distribution [10], where Concorde has the best mean and lowest variance. 
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Fig. 1. Average tour Quality and Redundancy. 
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The mean and variance of GRASP and GE are almost identical and close to the mean and 
variance obtained by Concorde. Since GRASP has slightly better redundancy we used it as 
the subject for the next experiment. 

3.3 Experiments with Record Keeping 

Under the IHC algorithm, the generation of a tour that has been previously considered 
results in generating tours that have also been previously considered, and eventually 
leads to a local maximal tour that has been previously computed. Therefore, the 
obvious solution is to record all computed tours so that duplicates can be detected. In 
general, this is not feasible for large TSP instances, so we employ a cache mechanism 
to limit the size of memory used by the IHC algorithm. However, we first use 
unbounded memory record keeping on 50 random Euclidean graphs, of 100 vertices 
each, in order to determine an experimental upper bound on IHC speedup using 
record keeping. For IHC runs of 100,000 iterations, the average speedup using 
unbounded memory record keeping was 10.9. The same speedup is obtained with a 
64,000 blocks cache (cache of 896KB). Figure 2 shows the speedup obtained with 
different cache configurations [2, 8, 13]. It is apparent that even with a small cache 
size 16KB, we still achieve a speedup of 6.7. 



Speedup obtained with different cache 
configurations 




Fig. 2. Speedup obtained with different cache configurations. 



4 Conclusion 



In this paper, we evaluated the use of six different construction algorithms with the 
IHC method of solving the TSP. Our experimental results show that the greedy, 
greedy-jump, and GRASP construction algorithms all perform well. The greedy- 
jump construction algorithm also generates significantly fewer duplicate tours than 
the other algorithms, indicating that it may be more robust in situations where many 
local minima occur. Furthermore, we show that utilizing a record keeping mechanism 
modeled on cache memory is effective at improving IHC efficiency and can provide a 
speed up of up to 10. 9x in the given configuration. Moreover, even when the memory 
is only large enough to hold a small percentage of duplicate tours generated, cache 
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still provides a significant performance improvement. Thus, regardless of the 
complexity of a TSP configuration and memory limits, even a small cache is useful. 
This work has applications in the study of the traveling salesman problem and in 
combinatorial optimization problems in general. 

Future work involves further investigation of redundancy in the construction 
algorithms. We plan to investigate stochastic mechanisms to minimize redundancy 
and study its affects on algorithm performance and quality of the generated tours. 
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Abstract. We describe CP-Rota, a new constraint programming appli- 
cation for rotating workforce scheduling that is currently being developed 
at our institute to solve real-life problems from industry. It is intended 
to complement FCS, a previously developed application. The advantages 
of CP-Rota over FCS are a significantly smaller and more maintainable 
code base, portability across a range of different language implemen- 
tations and a more declarative approach that makes extensions easier 
and mistakes less likely. Our benchmarks show that CP-Rota is already 
competitive with FCS and even outperforms it on several hard real-life 
instances from the literature. 

Keywords: Staff Scheduling, Cyclic Schedule, Manpower Scheduling, 
Timetabling. 

1 Introduction 

Computerized workforce scheduling has interested researchers for over 30 years. 
To solve rotating workforce scheduling problems, different approaches have been 
used in the literature, including exhaustive enumeration ([5], [5j), constraint 
(logic) programming, genetic algorithms ([5]) and local search methods. 

In the present paper, we describe CP-Rota, a new constraint application for 
rotating workforce scheduling that is currently being developed at our institute 
to solve real-life problems from industry. It is intended to complement FCS, a 
previously developed application that is currently commercially used in many 
companies in Europe. CP-Rota builds upon, contributes to and improves previ- 
ous constraint programming approaches to rotating workforce scheduling in the 
following ways: 

— CP-Rota is written in portable Prolog and will eventually be released under 
a permissive licence to benefit both researchers and practitioners. Much of 
its code is already available on request at the time of publication. 

— CP-Rota implements new allocation strategies (available as options for users 
to choose) that we discovered and discuss in this paper, which yield signifi- 
cantly improved performance on some real-life instances. 

— Our benchmarks on real-life instances show the tremendous potential of con- 
straint programming in rotating workforce scheduling, also and especially 
due to using different language implementations where they excel. 
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2 Related Work 

Many different approaches for solving rotating workforce instances are docu- 
mented in the literature. Balakrishnan and Wong l! solved a problem of rotating 
"workforce scheduling by modeling it as a network flow problem. Laporte [5] con- 
sidered developing the rotating workforce schedules by hand and showed how the 
constraints can be relaxed to get acceptable schedules. Musliu et al. [9j proposed 
and implemented a method for the generation of rotating workforce schedules, 
which is based on pruning the search space by involving the decision maker dur- 
ing the generation of partial solutions. The algorithms have been included in 
a commercial product called First Class Scheduler (FCS) 0], which is used by 
many companies in Europe. In [10j . Musliu applied a min-conflicts heuristic in 
combination with tabu search. Although this yields good performance on many 
instances, the resulting search method is incomplete and its results are there- 
fore not directly comparable with FCS. This paper also introduced 20 real-life 
problems collected from different areas in industry and the literature|j 

The use of constraint logic programming for rotating workforce scheduling was 
first shown by Chan in [3] . Recently, Laporte and Pesant [7] have also proposed 
a constraint programming algorithm for solving rotating workforce scheduling 
problems, implemented in ILOG and requiring custom extensions. 



3 The Rotating Workforce Scheduling Problem 

With CP-Rota and in the present paper, we focus on a specific variant of a 
general workforce-scheduling problem, which we formally define in this section. 
The following definition is from [9J and proved to be able to satisfactorily handle 
a broad range of real-life scheduling instances in commercial settings. A rotating 
workforce scheduling instance as discussed in the present paper consists of: 

— n: Number of employees. 

— A: Set of m shifts (activities) : oi, 02, . . . , a m . 

— w: Length of the schedule. A typical value is w — 7, to assign one shift type 
for each day of the week to each employee. The total length of a planning 
period isnxio due to the schedule's cyclicity as discussed below. 

— R: Temporal requirements matrix, an m x w-matrix where each element Ri j 
shows the required number of employees that need to be assigned shift type i 
during day j. The number Oj of day-off "shifts" for a specific day j is implicit 
in the requirements and can be computed as Oj = n — Yn=i ^d- 

— Sequences of shifts not permitted to be assigned to employees. For example, 
one such sequence might be ND (Night Day): after working in the night shift, 
it is not allowed to work the next day in the day shift. A typical rotating 
workforce instance forbids several shift sequences, often due to legal reasons 
and safety concerns. 



Examples available from http://www.dbai.tuwien.ac.at/staff/musliu/benchmarks/ 
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— MIN S and MAX S : Each element of these vectors shows, respectively, the 
required minimal and permitted maximal length of periods of consecutive 
shifts of the same type. 

— MIN-U, and MAX„,: Minimal and maximal length of blocks of consecutive 
work shifts. This constraint limits the number of consecutive days on which 
the employees can work without having a day off. 

The task in rotating workforce scheduling is to construct a cyclic schedule, which 
we represent as an n x w matrix S n-W eiU {day-off}. Each element Sij denotes 
the shift that employee i is assigned during day j , or whether the employee has 
time off. In a cyclic schedule, the schedule for one employee consists of a sequence 
of all rows of the matrix S. 

The task is called rotating or cyclic scheduling because the last element of 
each row is adjacent to the first element of the next row, and the last element of 
the matrix is adjacent to its first element. Intuitively, this means that employee i 
(i < n) assumes the place (and thus the schedule) of employee i + 1 after each 
week, and employee n assumes the place of employee 1. This cyclicity must be 
taken into account for the last three constraints above. 

In the present paper, we consider the generation of a single schedule that 
satisfies all the hard constraints given in the problem definition. Fulfilling all 
these constraints is usually sufficient in practice. The same constraints that we 
use in this paper are used in the commercial software FCS for generating rotating 
workforce schedules. This system has been used since 2000 in practice for many 
companies in Europe and the scheduling variant we discuss in this paper proved 
to be sufficient for a broad range of uses. However, FCS has several shortcomings 
that led us to consider constraint programming as an alternative approach. We 
discuss these shortcomings and their remedies with CP-Rota in the next section. 

4 Shortcomings of FCS and Their Remedies in CP-Rota 

Although FCS has been in commercial use since 2000 in several companies and 
proved to be an acceptable solution for many applications in practice, it has 
several disadvantages that hinder its further development: 

— The code base of FCS has gotten quite large and hard to maintain. This 
makes user modifications difficult and error-prone. New scheduling ideas 
cannot be rapidly prototyped but require substantial development effort. 

— FCS was implemented in Visual Basic and thus depends on essentially a sin- 
gle supported language implementation, which is in addition also not freely 
available. This complicates the sharing of code with other researchers and 
practitioners for joint development and turns every mistake in the language 
implementation into a potentially show-stopping problem. 

— From |10j , it is known that local search approaches - although incomplete - 
can significantly outperform FCS on some instances. Clearly, it is highly 
desirable to improve the running times of FCS to more closely match such 
competing approaches while retaining the completeness of the search. 
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When we started to work on the successor of FCS for the above reasons, we 
initially looked into constraint programming in the hope to significantly reduce 
the size of its code base. The promise of constraint programming was to just state 
the necessary requirements with high-level constraints and to then use built-in 
enumeration methods to search for solutions. 

We implemented the successor using the portable constraint programming 
model described in Section [5] and named it CP-Rota. Eventually (see Section [7J), 
CP-Rota even outperformed its predecessor on many instances, making con- 
straint technology a potential remedy for all of the above original shortcomings. 

5 A Constraint Formulation for Rotating Workforce 
Scheduling 

Our initial development environment was SWI-Prolog, which we chose due to its 
convenient libraries, tools and workflow, and also because it is freely available. 
We then ported the model to GNU Prolog due to its much better performance, 
and because it is also freely available, changes. When experimenting with cus- 
tom allocation strategies (Section [5)), GNU Prolog's lack of garbage collection 
hindered testing with larger instances, and we therefore ported the model also to 
B-Prolog, which is also a very efficient Prolog implementation and available free 
of charge for personal use. In all these systems, we model the rotating workforce 
problem as follows: 

— The schedule is represented as a list of lists, and each element is a finite 
domain variable that denotes the shift type scheduled for this position. 

— The temporal requirements are enforced via global_cardinality/2 con- 
straints on the columns of the schedule. In GNU Prolog, fd_exactly/3 con- 
straints are used instead. 

— The minimal/maximal-length constraints on consecutive shifts of the same 
type are enforced via automaton/3. 

— Reified constraints are used to map shifts of all types to either "work" or 
"day-off", and a second automaton/3 constraint is used on these reified 
variables to limit the number of consecutive work and day-off shifts. 

— Reified constraints are also used to express forbidden patterns. For example, 
if "0 4 3" is forbidden, the constraint is: 

X k #= #/\ X k+1 #= 4 #==> X k+2 #\= 3 

for all variables Xk, also taking into account the schedule's cyclicity. In 
B-Prolog, not in/2 (negated extensional) constraints are used for better 
performance. 

It only took a few days to implement this basic model (700 LOC, including a 
50 LOC parser for instance specification files and 50 LOC for visualisations) 
and to get it to run on all of the above Prolog implementations. Only built-in 
constraints are used in all systems. In contrast, the development of FCS took 
several months. 
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6 Labeling and Allocation Strategies 

The default strategy in CP-Rota is to first label the (reified) work/ "day-off " 
Boolean variables. Then, all original variables of the schedule are labeled with 
the "first- fail" option. We call this strategy Si . When S\ did not yield a solution 
within fOOO seconds, we used Strategy S2, which is to label all schedule variables 
from left to right, trying their values from lowest to highest. If this does not 
yield a solution within 1000 seconds, S3 is used: Reified constraints are used 
to compute, for each column, the number of still missing shifts of each type. 
Processing the columns in order, we then choose the shift type that misses the 
least number of elements in that column, and assign it to a feasible variable with 
smallest domain. When S3 also fails to find a solution within 1000 seconds, Si 
is used: It is similar to S3, except the columns are not processed from left to 
right, but in descending order of their number of still missing shifts of any type. 

Table 1. Comparison between FCS and CP-Rota. 
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7 Comparison with the Commercial System FCS 

Table [T] compares the performance of CP-Rota with that of FCS on 20 real- 
life instances from [10]. To the best of our knowledge, FCS is a state-of-the-art 
commercial system for generating rotating workforce schedules. We tested all in- 
stances on a Pentium 4, 1.8 GHZ, 512 MB RAM, using the latest versions of FCS, 
GNU Prolog (1.3.1) and SWI-Prolog (5.9.10). Except where stated otherwise, 
timing results are from GNU Prolog. 

The table shows that CP-Rota nicely complements its predecessor so that 
3 more instances than previously can now be solved. On 7 instances, CP-Rota 
outperforms FCS already with its default strategy (Si), the converse holds for 
6 instances. 

8 Future Work 

Future improvements to CP-Rota include the addition of a more convenient user 
interface, real-time interaction with decision makers and the implementation of 
additional allocation strategies. 
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Abstract. Online Dispute Resolution (ODR) tools are being seen as a way to 
settle disputes out of courts, namely under virtual environments. However, the 
acceptance of these alternative methods is still relatively restricted, once exist- 
ing tools are relatively undemanding anddomain-centered.Indeed, there is the 
need for conceptual tools whose parts may be arranged for particular use, i.e., to 
operate in different domains. Following this line of attack, in this paper it will 
be presented a new agent-based approach to ODR.lt comes in an abstract and 
formal form, in order to beindependent of the legal domains, but specific 
enough to be applied to concrete ones. The main advantage is that functionality 
reuse is maximized, making architectures simpler to implement and to expand. 

Keywords: Multi-agent Systems, Online Dispute Resolution, Intelligent 
Environments. 



1 Introduction 

The technological evolution led to unprecedented changes in virtually every aspect of 
the society. As an example, a significant percentage of contracts in the most different 
fields is now signed under an electronic or virtual environment, whereas in the past all 
contracts were signed with the physical presence of all the parties involved. Another 
difference is that in the past these contracts were paper-based, while now they are 
completely electronic, without a mandatory paper support. The most evident example 
of this new reality is visible on the recent e-commerce phenomenon [1], of which 
web-sites like the ones of Amazon and e-Bay are the best examples. However, simi- 
larly to traditional paper-based contracts, disputes can also arise in an electronic set- 
ting. In fact, according to Javelin Strategy and Research 1 , only 45% of electronic 
costumers are satisfied with the quality of products sold online. It is nowadays clear 
that the traditional approach for solving disputes by means of litigation in court is not 
feasible anymore. 

An alternative way of solving disputes arising out of electronic contracting is 
Online Dispute Resolution (ODR), allowing already traditional alternative dispute 
resolution methods such as negotiation, mediation or arbitration to be moved "from a 
physical to a virtual place" [2], ODR makes availableto all the parties an undemand- 
ing way of litigation, and a simple and efficient way to deal with disputes, thus saving 
time and assets [3]. 



https://www.javelinstrategy.com/research 
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In devising such systems, we are taking into consideration the Katsch/Rifkin vision 
of the four parties in an ODR process: the two opposing parties, the third neutral party 
and the technology that works with the mediator or arbitrator [4] .We clearly assume a 
gradual tendency to foster the intervention of autonomous software agents, whichac- 
teither as decision support systems or as real electronic mediators [5]. The considera- 
tion of this wider role for software agents is based in the use of Artificial Intelligence 
based methodologies and techniques eitherfor problem solving or knowledge repre- 
sentation and reasoning, such as Case-Based Reasoning [6]. 

In this paper we focus on the problematic of developing an architecture for multi- 
domain dispute resolution, whose objective is to enable a range of services targeted at 
assisting the disputant parties, independently of the domain of the dispute. This archi- 
tecture is abstract in the sense that it encompasses perceptions that are common to the 
several domains addressed. It is also specific in the sense that it provides support for 
the specificities of each domain, in a transparent manner. Indeed, the objective of this 
work is to develop an agent-based architecture that can be used in several legal do- 
mains. It is being developed in the context of the Portuguese legal system, covering 
three major fields of The Law: Family Law [13], Consumer Law [12] and Labor Law 
[11]. 

2 Identifying Abstract Perceptions and Processes 

In a dispute resolution process we can talk of abstract entities that are present regard- 
less the domain of the dispute. As an example, to a certain point, a negotiation process 
will always be a sequence of rounds in which, in each round, each party states itsview 
about the current proposal on the table, i.e., the process goes on independently of the 
subject of the proposals. The same happens with certain perceptions. As an example, 
independently of the domain of the dispute, a party will always be interested in know- 
ing its best and worst possible scenario. Therefore, in the development of an abstract 
architecture, firstly one has to identify which perceptions and processes are present in 
the quite a few domains that can be modeled. 

2.1 Abstract Perceptions 

On determining the abstract picture for a multi-domain ODR tool aimed at assisting 
the parties, one mustpursue by determining which information would actually be 
useful for the parties and then state the overall perception of the situation. Undenia- 
bly, it would be interesting for a party to determine to which extent is it reasonable to 
engage in a dispute resolution process, i.e., would a better outcome be reached using 
an alternative dispute resolution process instead of litigation? 

The thought that encompasses this is the BATNA - Best Alternative to a Negoti- 
ated Agreement, or the possible best outcome "along a particular path if I try to get 
my interests satisfied in a way that does not require negotiation with the other party" 
[7]. This conception is abstract as it is useful for any dispute, and it is constructive for 
parties once, at least, it contributes to the acknowledgement that an agreement may 
be disadvantageous [8]. A similar hypothesis is the Worst Alternative To a Negotiated 
Agreement (WATNA) [9]. A WATNA intends to estimate the worst possible 



An Agent-Based Architecture for Multifaceted Online Dispute ResolutionTools 91 

outcome along a litigation path. It can be quite relevant in the calculation of the real 
risks that parties will face in a judicially determined litigation, imagining the worst 
possible outcome for the party. 

However, it couldalso be interesting to consider the roomalready settled between 
the BATNA and WATNA, as a useful element to be taken into account for making 
(or accepting) a proposal. Of course, excluding the space between BATNA and 
WATNA, less dangerous is for the party not to accept the agreement (unless, of 
course, its BATNA is really unfavorable). A wider room between BATNA and 
WATNA would usually mean that it becomes rather dangerous for the party not to 
accept the ODR agreement (except in situations when the WATNA is really not in- 
convenient at all for the party). This ideais evidently related to the Zone of Possible 
Agreement (ZOPA) proposed by Raiffa (1982) [10]. 

Moreover, it would be interesting for a party to understand the region of such a 
space in which aresult is more likely to come about, i.e., if the parties are going to 
solve the dispute through litigation, which is the most likely outcome? In fact, stick- 
ing only with the BATNA and WATNA may not be realistic. Thus, an informed party 
should also consider concept of MLATNA - Most Likely Alternative to a Negotiated 
Agreement. Using the same arguments, it may bring to a close that the existence of 
metrics that assess the probability of each possible outcome could also be extremely 
useful for a party. Thus, it will be possible to consider the view of plausible case: a 
possible outcome with an associated value of likeliness. 

2.2 Abstract Processes 

Likewise, there are processes that implement key functionalities, which are intangible 
enough to be reused in a number of legal domains. Concretely, two abstract processes 
were applied: case retrieval and negotiation. The former caters for a selection of a set 
of past known cases that can be relevant, considering the current dispute. Parties can 
then analyze these cases, and obtain valuable information about past similar disputes. 
The remaining one is concerning with negotiation, making available a common nego- 
tiation process in which two or more parties state their opinions in sequential rounds 
about the subject being negotiated. This process allows the parties to reach a consen- 
sus about the outcome of a dispute. 

3 An Architecture for a Multifaceted ODR Platform 

Let us now depict the architecture that implements the ideas described so far. There 
are two main issues: the actual abstract architecture and an ontology that allows for 
the agents to perform specific tasks according to the domain of discourse. 

3.1 The Architecture 

The architecture builds on a set of software agents, autonomous and proactive, that 
cooperate in order to achieve their goals. On the other hand, a development strategy 
was devised that organizes the agents in two categories, namely: high-level agents 
and low-level ones. High-level agents perform tasks that do not need explicit domain- 
dependant information. Low-level ones are closer to the legal domain, inneedof 
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methodologies for problem solving and/ormethods for knowledge representing and 
reasoning, in particular to think up the legal procedures. In general, high-level agents 
monitor and set the behaviour ofthe low level ones. Low-level agents act on the ob- 
ject-level data and knowledge, that make the extensions of the functions that describe 
the universe of discourse. 

Let us take as example anegotiation process:high-level agents guide the process 
and determine when a new round should start, or finish; low-level ones have the 
autonomy to choose the actions to be performed, according to the state of the domain 
of discourse. That is, depending on the domain, low level agents will compute a valid 
output in each round. All the agents and their roles are depicted in [11]. These agents 
were defined according to the specific requirements of the project, following an itera- 
tive cut-down practiceof increasing specification. 

3.2 The Ontology 

Having specified the high level architecture, let us now show how it can be used to 
implement specific services. This will be done through the use of ontologies. The 
method consists in defining a specific ontology for each legal domain. Each ontology 
encodes a domain theory, actions, constraints and rules, revealing to the agent what 
action is to be executed and how to achieve it, i.e., according to the legal domain 
being attended to. The main advantage of this approach is that a single agent can be 
used to perform a similar task in a wide range of domains in opposition to a tradi- 
tional one in which an agent would be used for each different domain. As an example, 
let us consider the action of searching for similar cases. Instead of having three differ- 
ent agents, one for each of the legal domains under equation (i.e., Commercial, Fam- 
ily and Labour), we have one single agent and three different ontologies. This line of 
attack significantly increases functionality reuse and allows for a single architecture 
supporting services in a wide range of domains. It also simplifies the task of extend- 
ing the architecture for addressing new legal domains by developingan ontology for- 
everyone, with all the actions, rules, constraints and specific theories. This will tell the 
low-level agents how to act in a new domain. 

Each ontology comprises four components: the vocabulary, the actions, the fea- 
tures and a theory. The vocabulary contains all the words that can be used to describe 
the entities that belong to the domain. Actions define how each action should be exe- 
cuted according to the domain of the ontology. The theory will define all the elements 
that make up the perceptions about the universe of discourse (e.g., type, number). 
Finally, features allow for add up of invariants to the perceptions and actions that 
make up the ontology. 

As an example, let us consider the computation of the BATNA and WATNA in 
two different legal domains, namely: Commercial Law and Labour Law. These 
twoabstract conceptions denote, as it was described above, the best and worst possible 
scenario in a litigation process, i.e., it has a meaning that can be of use independently 
of the domain of the dispute. Nevertheless, depending on that domain, it is computed 
in different ways. In Labour Law we will have to consider views in terms of worker 
antiquity, monthly salary, seniority, a joust cause for dismissal, just to name a few. On 
the other hand, in Commercial Law key views would be the date in which the product 
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was bought, the type of the product, the type of the warranty, the state of the product, 
among others. Thus, two different ontologies (i.e., Labour and Commercial) define the 
act of computing the BATNA and WATNA, in different ways: 

A simplification of the rules that allow the computation of the BATNA and WATNA for the 
Portuguese Labour Law, as it is given in Decree of Law (DL) 7/2009 (Portuguese laws).. 

Def_Rule 396 

if RULE_394 then 

WATNA := 3 * (M_SALARY + SENIORITY) 
if TEMPORARY_CONTRACT then 

if WATNA < M_REMAINING * (M_S ALARY + SENIORITY) then 
WATNA := M_REMAINING * (M_S ALARY + SENIORITY) 
if WATNA < 15 * (D_SALARY + SENIORITY) then 

WATNA := 15 * (D_SALARY + SENIORITY) 
BATNA := 45 * (D_SALARY + SENIORITY) 
if BATNA < DAMAGE then 

BATNA := + DAMAGE 

A simplification of the rules that allow the computation of the BATNA for the Commercial 
Law, as it is given in DL 67/2003. 1 this example rule it will be considered only numbers 1 to 4 
of Article 5 th . 

Def_Rule5 

if IS_MOBILE then 

if DEFECT_COMPLAINT_DELAY < 60 then 
if WARRANTY_DELAY < 73 then 

BATNA := {"product repair in 3 days"; 
"product replacement"} 
else BATNA := {"no indemnity due"} 
else BATNA := {"no indemnity due"} 
else 

if DEFECT_COMPLAINT_DELAY <3 65 then 
if WARRANTY_DELAY < 1810 then 

BATNA := {"product repair in reasonable time"; 

4 Conclusion 

One of the major challenges that Online Dispute Resolution faces is the lack of tools 
that can address more than one legal field. In fact, most of existing tools focus only on 
small and very limited domains. Following the approach presented in this paper, three 
prototypes focusing on different legal domains are being developed. These prototypes 
are based on a single agent-based architecture targeted at providing services for 
dispute resolution. The services implemented are abstract enough to be useful in sev- 
eral legal domains and contain the specific rules needed to be used in each specific 
domain. Using ontologies to specialize a single agent in several legal domains makes 
the architecture simple and makes it easy to expand it to other legal domains, by add- 
ing new ontologies. This, we believe, will foster the development of multi-domain 
Online Dispute Resolution Platforms. 
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Abstract. In this paper, we propose a knowledge management system that em- 
ploys agents and case-based reasoning to extract tacit knowledge and support 
organizational learning and decision making at the level of individual workers, 
group workers and the whole organization. Agents will perform the tasks of in- 
quiry, investigation, sharing and updating of knowledge bases, including previ- 
ous similar cases, in order to help in finding the required knowledge for a 
specific problem. We have applied the system to develop a banking training as- 
sistant system and have run several experiments to measure a number of func- 
tionalities such as reliability, efficiency, the impact of weight parameters on 
extracted results, and the degree of match between the results produced by our 
system and the opinions of randomly selected experts in the chosen application. 

Keywords: Knowledge Management, Case-Based Reasoning, Multi-Agent. 

1 Introduction 

Knowledge is considered as a critical component of organization competency and an 
essential asset that requires careful management so that organizations can enhance 
and facilitate the decision making process [6]. We distinguish between explicit and 
tacit knowledge. Explicit knowledge could be explored and easily shared with others 
using documents, databases, and so on [1]. Tacit (embodied) knowledge is embedded 
inside a person's mind, therefore it is difficult to extract and share with others [4]. 
Organizational Knowledge/Memory (OM) refers to more than just databases and 
information systems involved in an organization. It encompasses processes and peo- 
ple. Decision makers (DM) in an organization often need to employ both explicit and 
tacit knowledge that are related to the specific problem which needs to be solved. The 
study of OM seeks to understand how knowledge is created, stored, accessed, trans- 
formed, handled and used effectively. Knowledge Management (KM) should provide 
relevant knowledge to assist processes in performing knowledge intensive tasks [12]. 

A Knowledge Management System (KMS) should involve providing knowledge, 
experiences and insights at the right time, to the right place, in the right form to the 
right knowledge worker [9, 10, 12]. These features characterize intelligent agents. 
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With agents, it is possible to capture the representation, coordination, and cooperation 
between heterogeneous processes and other agents. An appropriate KMS should have 
the ability to extract proper tacit knowledge when requested to do so [9, 10, 11] and 
has to support the process of DM and Organizational Learning (OL). 

In this paper, we propose a KMS that employs agents and Case-Based Reasoning 
(CBR) to extract tacit knowledge and support OL and decision making at the level of 
individual worker, group workers and the whole organization. Agents will perform 
the tasks of inquiry, investigation, sharing and updating of knowledge bases, includ- 
ing previous similar cases, in order to help in finding the required knowledge for a 
specific problem. We have applied the system to develop a banking training assistant 
system and have run several experiments to measure a number of functionalities such 
as reliability, efficiency, the impact of weight parameters on extracted results, and the 
degree of match between the results produced by our system and the opinions of ran- 
domly selected experts in the chosen application. Section 2 will be concerned with 
Multi-Agent Systems (MAS), OL and CBR. In section 3 we present an agent-based 
learning KMS. Section 4 will be concerned with data collection and implementation 
of a training assistant system for the central bank of Jordan. 

2 MAS, OL and CBR 

Agents are specialized problem solving entities with well-defined boundaries with the 
ability to communicate with other agents. They are designed to fulfill a specific pur- 
pose and exhibit flexible and pro-active behavior. They are autonomous in the sense 
that they can operate on their own without the need for guidance. They have control 
both over their internal state and over their actions. 

A MAS can be defined as a collection of agents with their own problem solving 
capabilities and which are able to interact among them in order to reach an overall 
goal [5], KM is mainly concerned with using, spreading, sharing, representing and 
storing the available knowledge of an organization. OM is about how to collect, store, 
and provide access to experiences and skills of both stored records and tacit knowl- 
edge. OL is concerned with enhancing the organization problem solving process [13]. 
In [3], a model of learning is proposed. It includes three learning levels: individual, 
group and organizational and two routes of flow: upward (individual to organiza- 
tional) and downward. This suggests that knowledge be structured to capture the three 
learning levels. 

CBR is a mechanism for solving new problems based on the solutions of similar 
past problems [7]. CBR supports many of the activities of KM and has been fre- 
quently proposed as a methodology for KM applications. CBR has been employed in 
the design of many KM systems [2]. 

3 Towards an Agent-Based Learning KMS 

An important feature of a KMS is Knowledge Communication (KC). KC allows for 
making knowledge accessible to and reusable by its different components [10, 11, 
12]. We employ four types of intelligent agents: (1) Knowledge Cooperative Agent 
(KCA) that will be responsible for analyzing and searching the Knowledge-Bases 
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(KBS) of the OM; (2) Reasoning Agent (RA) that will be involved with CBR tasks; 
(3) Learning Agent (LA) that will be responsible for modifying the OM and Task 
Agent (TA) that will be responsible for identifying when and how to apply KC. TA 
will be responsible for identifying to what level a specific problem belongs, whether 
in the CBR-Base or the KB. RA will investigate the CBR-base to identify similar 
cases to the given problem, and then RA will find percentages of similarities to cases 
and give the results back to the TA. If a desired case is founded then no need to call 
KCA. Otherwise, TA will call KCA in order to find the appropriate experts who may 
give a consultation about the problem. 

The different components of the system are illustrated in Figure 1 below. 
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Fig. 1. A Learning KMS. 



OM is divided into two parts: one for KBs of related entities and the other for 
CBR-Bases. Figure 2 illustrates the structure of an OM. There are three knowledge 
levels. An ontology is employed to structure the case bases into hierarchies (cf. Figure 
3). It stores an abstraction and the mapping of all domain-specific concepts and mean- 
ings and can be used to verify the existence of all acquired concepts in KB. A similar- 
ity measure is defined in order to retrieve similar cases. For each attribute of a given 
case and a given query, a similarity can be calculated. The attribute similarities can be 
combined to an overall similarity value. The employment of similarity measure makes 
it possible to rank the solutions. iiA case represents a previous experience of a situa- 
tion. It consists of two parts: a problem description together with the con- 
text/environment in which it has occurred and a solution to this case. KB can be ac- 
cessed during the retrieve phase using a similarity based measure. When presented 
with a particular case, DM calls TA in order to identify the type/level of problem 
based on its parameters. Weights can be associated with parameters to indicate their 
importance. Weights help in fine-tuning and reaching more accurate decisions during 
reasoning by enabling RA to retrieve the most similar and relevant cases. TA will 
generate a unique problem-ID and store the problem in the problem-base. RA can use 
the problem-ID and ontology in order to investigate the level to which the problem 



98 



M. Altawil, A. Moubaiddin, and N. Obeid 



belongs in the CBR-Base. The results are then passed on to TA to be studied in order 
to determine whether or not there is a need to involve KCA whose task is to finds the 
appropriate Knowledge Expert/s (KE) that can deal with the task. The process in- 
volves determining the appropriate knowledge level that contains similar solutions for 
the current problem. TA then presents the solution case to the KE for approval. If 
approved, LA will add the new case at the appropriate level. 
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Fig. 2. Structure of OM. 




Fig. 3. Case Base Structure using Ontology. 



4 Data Collection and System Implementation 



The Computer Department (CD) in The Central Bank of Jordan (CBJ) has to provide 
the technical support for the functional operations carried out at CBJ. The Training 
Division (TD) in the CBJ has to regularly prepare the Training Need Matrix (TNM) 
for CD. Some training bodies may make offers regarding their current and future 
training courses including their specifications. 

We have collected: (1) 1000 precious individual-cases, (2) 150 previous group-cases, 
(3) 50 organization-level previous course cases, (4) information/knowledge about 45 
individual employees's job such as names, descriptions and responsibilities and (5) 
information about Groups' s jobs such as names, descriptions and responsibilities. 
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We have used PL-SQL developer Ver. 6.0.3.893. as our Multi-Agent development 
tool for developing TA, RA, LA and KCA. Reader may refer to 
http://www.ju.edu.jo/sites/Academic/obein/Lists/Academic%20Conferences/AllItems. a 
spx for the pseudo-code of the agents.. We have run several experiments where each is 
designed to measure a specific functionality as follows: 

1. Test-Retest Reliability: The reliability of the model depends on reliability of the 
agents. We have measured the reliability of TA, RA, KCA and LA. 

2. Measuring Internal Consistency Reliability for RA: We measure internal consis- 
tency reliability for RA with regard to a single parameter. Figure 4 shows the val- 
ues of internal consistency reliability for 5 trials. The minimum value is 80%. 

3. Measuring System Efficiency: We have ran experiments to measure the efficiency 
of RA and KCA. We increase the number of cases and experts in the following or- 
der (1000, 2000, ..., 6000) for the case-base and (45, 90, 135, 180, 225, 270) for 
KB. Figure 5 shows the additional execution times RA needed to investigate the 
new cases. Figure 6 shows that the performance RA varies in a stable manner when 
the number of cases of additional organizations increases, (cf. Figure 7). 
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Fig. 4. Measuring the internal consistency 
reliability for RA. 



Fig. 5. Execution times that RA needs 
to investigate new cases. 
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Fig. 6. Variance values of execution vs 
Org. levels & average mean. 



Fig. 7. Variance to average mean vs 
variance to total mean. 



Fig. 8. Extracts by RA as course_name increases. 
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4. Measuring the Impact of Parameters' Weight on Results: We have run 10 dif- 
ferent trials, on each parameter while decreasing its value such as 25, 20, ..., for 
course_name of a random problem. The other parameters are kept fixed. Figure 8 
above shows the results that RA extracts are affected by the weight of 
course_name. When the weight increased the number of extracted cases increased. 

6 Concluding Remark 

We have, in this paper, proposed a KMS that employs agents and CBR to extract tacit 
knowledge and support OL and decision making. The experimental results, using 
various measurement methods, are reliable and efficient. We compared the results of 
the system with that of a committee of 7 experts from CBJ. The results were very 
positive. Out of 37 cases, there was a match on the 1 st choice of 23 cases, 12 on the 
2 nd choice and 2 on the 3 rd choice made by the system. Furthermore, the experts indi- 
cate that the system has indeed the ability to learn. 
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Abstract. In this paper innovative web admission auction-based algorithms are 
presented combined with load-balancing and QoS characteristics. Performance 
indexes as income of the web service or client satisfaction is discussed. Simula- 
tion experiments have been performed to validate the proposed mechanisms. 
They showed benefits of our developments. 

Keywords: auction mechanisms, admission control, load-balancing, scheduling 
algorithms in web service, quality of service. 



1 Introduction and Background 

Web services are used to provide the increasing volume of data in the computer net- 
work systems. Web clients are waiting for delivering data in a proper time, i.e. within 
the bounded values of the time delay. In many web services it is required to pay 
charges for utilizing their resources. Such services should be equipped with efficient 
admission control algorithms providing right and guaranteed service level (Quality of 
Service, QoS) to their users. 

In the literature there are various approaches to the problem of controlling the dis- 
tribution of service resources to the clients e.g. [3], [4], [6], [8]. The most common 
idea is based on the FIFO scheduling, so that resources are assigned to users accord- 
ing to the earliest time stamp of reported requests (offers). Other proposed scheduling 
policies taken into consideration are: the mean time of serving requests known from 
the history or distinguishing some requests classes according to requested resource 
size. The aspects of service level guarantee in web services were discussed in [3], [9] 
as a resources delivery in a proper time or as transfer parameter (bandwidth) assuring. 

The auction-based algorithms were also presented in many knowledge domains, 
i.e. for energy resources management, for radio spectrum selling, for bandwidths or 
telecommunication goods management [2], [6], [7]. Proposed systems do not concen- 
trate on serving the single users, thus they are oriented on providers and often use 
agent systems. 

Modern web services are designed for business profit. They try to apply some mar- 
ket mechanisms to solve scheduling problems and to serve user requests according to 
market prices in such a way to increase the benefit of web service. This may be done 
by application of auction algorithms which are aiming for buying and selling of goods 
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through a dynamic pricing strategy. Various multi-access and multi-functional auction 
systems were designed and implemented, including e-Bay, Allegro or Merkato. 

Several researches had tackled with the problem of decision making based on a 
specific auction policy named Vickrey auction. The basic idea of this kind of auction 
algorithms is that the price, which should be paid by the winners, is lower than de- 
clared [1], [6], [7], [10], [11]. 

In this paper two new auction-based scheduling algorithms were introduced and 
Vickrey mechanism for pricing offers was applied. The paper considers a combined 
approach of auction algorithm, QoS and admission control of requests, related to 
individual clients. The main contribution of this paper is an application of new 
mechanism of auction based on admission control of user requests arriving to the web 
service. Our proposal is featured with a charging schema in respect with acceptable 
QoS levels and closely connected with a client satisfaction. Proposed algorithms 
guarantee the time delivering of service resources (i.e. multimedia files, audio books 
files, books as text file) [10]. 

The process of offers collection from web clients will be discussed in the second 
section. The third section formulates the problem. In the fourth section the auction 
based scheduling mechanisms will be proposed. The fifth section presents simulation 
results. In the last section the conclusions are given. 

2 User Offers 

The role of web service will be to schedule web client requests and to guarantee them 
defined service level (defined bandwidth) for downloading files. Every user (web cli- 
ent) will be obtained to reveal his offer. Every offer (o n = {rj ,p uik , b uik } n ) will be char- 
acterized by triple of following variables: resource name, price, and bandwidth. 

Let's analyze an example: offer Oi=[r b 25$, 128 kb/s] includes resource r { for 25$ 
for bandwidth 128 kb/s, offer o 2 =[ti, 40$, 192 kb/s] includes resource r, for 40$ for 
bandwidth 192 kb/s. Allowable is to report many offers for one resource by one cli- 
ent, but the declared price for increasing number of bandwidth pieces should reflect 
non-decreasing valuation function. 

3 Problem Formulation 

The aim of the web service will be to manage of the bandwidth pieces assuring as- 
signed transmission speed. The service will support delivering of data files, for which 
web client will be obligated to pay a charge related to requested bandwidth. 

The problem will be formulated as follows: 
For given: 

- 0\, ...., o N - the set of offers, each offer contains the resource name, price and auc- 
tioned bandwidth, 

- R - the set of data files (resources), r, e R, 

- T - scheduling time interval, 

- B — total available bandwidth, 
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- the performance index defining an income of the web service (sum of submitted 
offers), 

a scheduling admission strategy should be found to select the set of offers Sd for proc- 
essing, which increases the performance index and assures the client satisfaction si- 
multaneously in every time instant (T). In this paper only the situation, in which 
server resources are not sufficient to satisfy all users, because of the fact that the total 
sum of expected bandwidth exceeds the total volume of bandwidth offered by the web 
service, is considered. 



4 Algorithms 

The managing process should increase the income of the web service (the sum of ser- 
viced offers) and should satisfy the web clients. Two scheduling algorithms will be 
discussed and examined. The first one - class-based auction algorithm is more justi- 
fied and the second one - profitability based auction algorithm based only on pro- 
posed offers. 

The first algorithm guarantees, that some offers from every request for every class 
will be selected in respect of the just treatment and customer satisfaction . The later 
policy does not consider class based selection and can lead to postponement of large 
files because of the same price both for small and large files. In both cases also a 
modified criterion could be defined, taking in account the client satisfaction. It could 
be e.g. the price paid by users (charge). When the client will pay finally the lower 
price than declared, he will be satisfied [1], [8], [9], [10]. The scheduling algorithm 
will run in following periods, which are defined by the periodic instances of the sche- 
duling interval T (i.e. every 30 min). 

4.1 Class-Based Auction Algorithm 

In this approach, it is assumed, that every resource belongs to one class [3]. The 
classes are determined by the data size, i.e. three classes are there distinguished: 

- the bronze for data size less than 10MB, 

- the silver for data size betweenlOMB and 50 MB, 

- the gold for data greater than 50MB. 

The next assumption is that the defined piece of total bandwidth is assigned justify for 
all classes, it means that from every class some offers will be served. 

The rest will be assigned to offers containing the maximal value of profitability in- 
dex. The approach algorithm consists of three phases. 

Phase 1. The first phase of the algorithm based on the clear rules - to every class 
some bandwidth will be assigned. Let's assume that 60% of the total bandwidth is 
assigned for all the requests classes. 

We consider three classes, so that, to every class 20% of total bandwidth will be 
assigned. For every class the offers with the highest price will be scheduled until the 
bandwidth for class is available. 
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The third assumption is that every class has a specific bandwidth, i.e.: 

- bronze class (128kb/s), 

- silver class (256 kb/s), 

- gold class (512 kb/s). 

Phase 2. The second stage of this algorithm takes a profitability metrics into 
consideration. For every offer an index will be calculated and then the offers with the 
highest value will be scheduled until the bandwidth is given. The profitability metrics 
tp„ can be defined as follows (1): 



where 

*££) (2) 

t n - time of transfer j-f/i resource for bandwidth b uk (2). 

The classification to some classes in the second stage doesn't matter. 
During this stage following set of offers will be chosen (3): 

L 

S d =[o n }=>max^(p n (3) 

and by assumption (4) 

f t b Hik <40%B (4) 

;=i 

Phase 3. The third phase is used to serve the dynamic requests. In case, when during 
the defined scheduling interval T some requests are rejected, the free bandwidth will 
be assigned to dynamic offers. The price will be calculated basing on the individual 
offer and requested bandwidth. The assigned bandwidth will base also on separately 
negotiation process. The third phase takes place only during the scheduling interval 
(when < dynamic scheduling time < 7). 

4.2 Profitability-Based Auction Algorithm 

This approach is based only on the value of profitability index defined in the second stage 
of the class-based auction algorithm. The following set of offers will be chosen (5): 

L 

S rf =K}=>max2>„ (5) 

i=i 
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and by assumption (6) 



llK^ B 



(6) 



The class division is not taken into consideration. The second phase is similar to the 
third phase presented in the class-based auction approach. 

Some algorithms were developed to increase the income of web service and for the 
client satisfaction, and were based on the second-highest price (Vickrey) approach. 

5 Results of Simulation 

Three algorithms: Class-based Auction Algorithm, Profitability based Auction Algo- 
rithm and FIFO Scheduling Algorithm were implemented in the Matlab environment. 
The stream of offers was simulated as a Poisson Process. For different number of of- 
fers, and different available bandwidths (10Mbps and 62.5 Mbps) the profit of web 
service was measured and compared. 

Fig. 1 and Fig. 2 present the performance index defined as the sum of the income 
of web service vs. number of serviced offers. The biggest profit (service income) was 
observed for Profitability based Auction Algorithm (depicted by squares) and a bit 
worse for Class-based Auction Algorithm (depicted by rhombus). The results for 
FIFO Algorithm (depicted by triangles) were always the worst. For the increasing 
number of offers the difference between Profitability based Auction Algorithm and 
Class-based Auction Algorithm was practically fixed whereas the FIFO algorithm 
stands out and the difference to others is increasing for growing number of offers. 




Fig. 1. Web service profit versus number of Fig. 2. Web service profit versus number of 
offers for 10 Mbps available bandwidth. offers for 62.5 Mbps available bandwidth. 



6 Conclusions 



In this paper two new scheduling algorithms based on auction approach were pre- 
sented. The classification of offers was depicted in an example. The profitability in- 
dex was shown. The idea of scheduling of offers was shown and discussed. Some fea- 
tures of presented mechanisms were distinguished. 
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The results of simulations were also presented. The experiments showed that the 
auction based scheduling process could be judged as very effective in the context of 
decision making in a web service with QoS. 

New approaches will be discussed, as offering more resources or the possibility of 
delivering data in later time. 

The later research will be focused on experiments in the environment of the IBM 
blade cluster system. 
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Abstract. This article presents an adaptive controller based on fuzzy 
rules emulated network (FREN) with the proposed on-line learning algo- 
rithm. The human knowledge about the unknown plant, 7- DOF robotic 
arm in this case, is transferred to be if-then rules for setting the network 
structure. All adjustable parameters are tuned by the on-line learning 
mechanism with time varying step size or learning rate. The main theo- 
rem is introduced to improve the system performance and stabilization 
through the variation of learning rate. Experimental system based on 
Mitsubishi PA- 10 is demonstrated the control algorithm validation. 

Keywords: Discrete-time; adaptive control; neuro-fuzzy; 7-DOF robotic 



1 Introduction 

To design the controller for robotic manipulators, dynamic and kinematic models 
usually are the normal requirements even those controllers has been constructed 
by artificial intelligence tools such as neural networks (NNs) and fuzzy logic sys- 
tems [TJ[2] ■ The system performance is commonly related on the model accuracy 
or approximated model in the case of using NNs to estimate robotic models. In 
this work, on the other hand, a direct adaptive controller based on fuzzy rules em- 
ulated networks [3] (FRENs) is introduced with out any requirement of robotic 
system modeling. According to an on-line learning, the close-loop performance 
can be guaranteed by using time varying learning rate. 

Recently, to ensure the system performance small learning rates or step sizes are 
often used in gradient search methods because of the system stability but the small 
step size can slow down the reaching solution [5] . In [5] , the suitable learning rate 
which can guarantee the system stability has been widely discussed but the using 
limit or operation range has been considered because of fixed variable learning rate. 

2 Robotic System with Non-affine Discrete-Time Systems 

According to design the controller with out any requirement of robotic system 
modeling, this section, we consider these robotic systems as a class of non-affine 
formulation as follows: 
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x(k + l) = f N (ui(k),u(k)), (1) 

when /jv( - ) is an unknown nonlinear function and fJ>(k) = [x(k) x(k—l) ■ ■ ■ x(k — 
n) u(k — 1) u(k — 2) • • • u(k — m)] T . According to some assumptions and the 
approximation mentioned in [3], the system ([T]) can be rearranged as 

x(k + 1) - F N (fi(k)) + G N (p(k))u(k). (2) 

In this work, let us assume that these nonlinear functions Fj^(k) and G]\[{k) are 
all unknown. The control effort u(k) is directly determined by FREN as 

u(k)=(3 T (k)4>(k), (3) 

when (3(k) G R l is an adjustable parameter vector and <fi(k) G R l is FREN's 
basis function vector where I denotes as the number of fuzzy rules. 



3 Parameters Tuning Algorithm 

In this work, only on-line leaning mechanism is applied with the associate of 
some designed parameters. The gradient descent method with the proposed time 
varying step size is introduced to adjust these parameter (3i for i = 1, 2, • • ■ , I. 
The cost function E(k), which is needed to be minimized, can be defined as 

E(k) = ie 2 (fc), (4) 

where e(k) — Xd{k) —x{k). At time index fc + 1, all adjustable parameters (3i can 
be determined by 

(3 i (k + l) = p i (k)- V (k) dE d i ^ k) 1 \ (5) 

when rj{k) is a time varying learning rate. In this work, we introduce the deter- 
mination method to obtain the possible biggest learning rate when the system 
stability can be guaranteed. 

Apply the chain rule through ^ and @, we obtain 

dE{k + 1) dE(k + 1) dx(k + 1) du(k) 



dj3i{k) dx(k + l) du{k) df3i{k)' 

= -[x d {k + 1) - x(k + l)]y p (k)<pi{k). (6) 

Thus, the tuning law can be rewritten as 

0i(k + 1) = 0i(k) + m(k)e(k + l)y p {k)4> t {k), (7) 

where y p (k) denotes g~?fe ■ Let us consider the system formulation in (0) again, 
clearly, we have 

y p {k) = G N { l i{k)). (8) 



Adaptive Fuzzy Rules Emulated Networks Controller 109 

4 Closed Loop Performance 

The key of this work is to determine the learning rate rj(k) for every time index 
k. Substitute the control effort u(k) given by ([3]) into the system formulation 
©, we have 

x{k + l)=F N (k) + G N {k)f3 T (k)cj)(k). (9) 

Thus, the next time step error can be rewritten as 

e(k + 1) = x d (k + 1) - F N (k) - G N {k)(3 T {k)ct>(k), (10) 

Substitute (jTUJ) into (JT]), the adaptation law can be obtained as 

f3(k + 1) = [I - V (k)G 2 N (k)cf, T (k)cj,(k)}f3(k) + V (k)[x d (k + 1), 

-F N (k)]y p (k)(f>(k). (11) 

For the convenient presentation, let us select learning be 

r](k) = =2 , (12) 

G^ T (fc)0(fc)' 

when G u is the positive upper bound of «„ ' as given by assumption [2b 
and < 7 < 2 is the designed parameter. 

Next, we introduce the proof of system performance based on the proposed 
controller with the following theorem. 

Theorem [2} System stability (Closed loop system convergence) 

Let the desired trajectory Xd{k) be bounded and the upper bound of Gj^(k) be 
known as G u . Determine the control effort u(k) by Q and tune parameters by 
with the varying learning rate given by (|12|) when < 7 < 2. Then the 
tracking error e{k) is bounded for the nonlinear system given by |T|). 
Proof. Let define the Lyapunov function candidate as 

V(k) = ie 2 (fc), (13) 

thus the change of Lyapunov function can be given as 

AV{k) =V(k + l)-V(k), 

= Ae(kMk) + ^}. (14) 

Let consider the next time index error can be written by e(fc + l) = e(k) + Ae(k), 
where Ae(k) can be estimated by Ae(k) k, A^ /3 (k)Af3(k), when A e ^(k) = 

i^sMw imw ■ • • ^wiwr^- B ? usin § the chain rule > we have 

de(k + l) de{k + 1) dx(k + 1) du(k) 



d0i(k) dx(k + l) du(k) df3i(k)' 

= -G w (fc)^(A). (15) 
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Thus, we obtain Ae(k) ss —Gjy (&)</> (k)A/3(k). From the tuning law obtained 
by (II ID . the change of adjustable parameters can be rewritten as 

Ap(k) = -r](k)G%(k)\\cl>(k)\\ 2 f3(k)+r,(k)[x d (k + l)-F N (k)}G N (k)cf>(k). (16) 

By using (|T6|) . we have 

Ae(k) = V (k)G%(k)\\cl>(k)\\ 2 [-x d (k + 1) + F N (k) + G N (k)<f> T (k)f3(k)], 

= -r,(fc)G 2 v (fc)||0(fc)|| 2 [ e (fc + l)]. (17) 

We can rearrange (fT7|) as 



-r,(k)G%(k)\\<Hk)\\ 2 e(k ) 
l + r)(k)G 2 N (k)\\<t>(k)\\ 






Substitute (JTHJ) into (TJJI, we obtain 

AV(k)= -v(.k)G%(k)\\cj>(kWeHk) r v (k)G%(k)\\<t>(k)\\ 2 ] nq) 

[> l + V (k)G%(k)MkW L 2 + 2r / (fc)G 2 v (fc)||0(fc)|| 2 J- l ' 

With the learning rate give by (|12p . the change of Lyapunov function candidate 
can be rearranged by 



. 7[ OgM] 



- 7 [ g"( fc ) i 2 



1 + 7 



G JV (fc) 1 2 



^W = I T75^T2 1 - ~ rrJ^2 s e2 ( fc ) < °- (2°) 



2(1 + 7 r^W ] 2 ) 



D 



Remark: According to ((20|) . sign or direction of Gjv(fc) is not needed. 

The importance thing that we need to estimate is G u . With the following 
examples, the parameter G u is clearly designed and discussed. 

5 Experimental Setup and Results 

In this experimental setup, the proposed control algorithm is implemented to 
control a 7-DOF Mitsubishi PA-10 robotic arm system. The overall system con- 
figuration is illustrated in Fig. [TJ Seven FRENs are designed to control each of 
joints independently. This robotic system is operated in the velocity mode con- 
trol which means FRENs must generate the velocity commands to move very 
joints to follow the desired trajectory. 

To begin the controller design, the suitable IF-THEN rules are all needed to 
be specified. Based on the knowledge of PA-10, those IF-THEN rules can be 
given as the followings: 

If e(k) is PL Then uj\(k) = /3pi(fc)0i(fc), Large e in positive; Fast u in positive, 

If e(k) is PS Then (Vz(k) = /3ps(k)<j>2(k), Small e in positive; Slow w in positive, 

If e(fc) is Z Then u)z(k) — /3z(k)(ps(k), Reach desired position; Stop moving !!!, 
If e(k) is NS Then u>4,(k) = /3Ns(k)(j>4,(k), Small e in negative; Slow u) in negative, 
If e(k) is NL Then ws(fc) = /3NL{k)<f>5(k), Large e in negative; Fast u in negative, 
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Fig. 1. PA- 10 FREN controllers configuration block diagram. 




Fig. 2. PA-10 Tracking performance qi(k) and ei(fc). 




Fig. 3. PA-10 Tracking performance qiik) and er(k). 
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when u> denotes the velocity command determined by FREN and e(k) is the 
position error defined by qd(k) — q(k) where qd{k) is the desired position at time 
index k. 

The tracking performance for every joints q\ and qi can be illustrated in Fig. [2] 
and O respectively, with theirs error signals. This experimental has been carried 
out for all 7 joints but we display only the first and seventh joints because of the 
page number limitation. 

6 Conclusion 

An adaptive control based on fuzzy rules emulated networks for 7- DOF robotic 
arm has been introduced in this article. The designer knowledge about the 
robotic arm including the physical limit has been directly integrated to FRENs 
through its IF-THEN rules and initial parameters. To control all DOF, the par- 
allel configuration of FRENs has been demonstrated with the closed-loop per- 
formance analysis. The variation of learning rate has guaranteed the system 
stability along the on-line learning phase. The system validation has been con- 
firmed by the experimental setup with Mitsubishi PA- 10 robotic arm. 
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Abstract. Non-deterministic reality is a severe challenge for autonomous robots. 
Malfunctioning actions, inaccurate sensor perception and exogenous events eas- 
ily lead to inconsistencies between an actual situation and its internal representa- 
tion. For a successful mission in dynamic environments, a robot is thus required 
to efficiently cope with such inconsistencies. 

In this paper, we present a belief management system based on the well-known 
agent programming language IndiGolog and history-based diagnosis. Extending 
the language's default mechanisms, we add a belief management system that is 
capable of handling several fault types that lead to belief inconsistencies. First ex- 
periments in the domain of service robots show the effectiveness of our approach. 

Keywords: belief management, diagnosis, history-based diagnosis, autonomous 
robot. 



1 Introduction 

There is an increasing interest to target autonomous mobile robots for complex tasks 
in dynamic (non-deterministic) environments. Related target applications range from 
simple transportation services via visitor guidance in a museum to autonomous car 
driving! 1 2|. The complexity of such application domains raises the demands regarding 
autonomous reasoning capabilities of deployable systems. Appropriate robots have to 
consider, for instance, a high and dynamic number of entities and objects, including 
their complex and spatial relations. In this context, inherent challenges a robot has to 
live up to are those of facing novel scenarios and having to cope with situations where 
a robot's assumptions about a situation are in conflict with reality. 

Imagine for instance a delivery robot whose task is to move objects between rooms 
and assume that it perceives some object that is known to be at some other place. Ob- 
viously, there is more than one explanation for this crucial inconsistency. For instance, 
(1) that the robot drove to the wrong room (execution fault), (2) a sensing fault, and (3) 
that someone moved the object (unobserved exogenous event). A robot that is unable 
to handle such inconsistencies would neither be able to successfully finish the given 
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task, nor to continue operation from a sane situation. Thus, in order to ensure quality 
of service, a reasoning mechanism and belief management is needed that is able to deal 
with this ambiguity by considering multiple hypotheses. 

For our implementation of such a belief management system, based on the agent pro- 
gramming language IndiGolog, we use the situation calculus and adapt history-based 
diagnosis [3| developed by Iwan and colleagues. Our system allows us to detect incon- 
sistencies, describe belief ambiguities, and generate multiple hypotheses as well as rank 
them. While we adopt the favorite hypothesis for immediate operation, we keep track 
of the alternatives in case the chosen favorite is proven to be wrong by future data. 

The remainder of this paper is organized as follows. Related work is covered in 
Section [2] In Section [3] we depict our belief management system, where section 13.11 
contains some preliminaries and Section [3T2l covers the details of our system. Experi- 
ments can be found in Section^ followed by conclusions drawn in Section|5] 



2 Related Research 

In the following we discuss previous work in three research areas that are relevant to 
our paper: (1) fault diagnosis, (2) hypothesis discrimination and (3) dealing with uncer- 
tainties in acting and sensing. 

Diagnosis, i.e. the detection and localization of faults, is a major topic in research. 
Many approaches like [4 5 1 specifically address the diagnosis of sensing and/or actuator 
faults. For complex and dynamic environments, classic diagnosis like consistency-based 
diagnosis [6] is unfortunately too static. While it blames faulty components, for dy- 
namic environments it is more appropriate to focus on correct action (event) sequences 
171 . In Q, Iwan describes a diagnosis approach for robots in dynamic environments 
that is based on the situation calculus, which we will discuss in Section [3~T1 

Once a set of diagnoses has been found, the correct one has to be isolated, or at least 
we have to identify the most probable one(s). In [8 1 the authors proposed a planning 
method to derive action sequences that improve the diagnosis quality by ruling out in- 
appropriate ones. Handling uncertainty in sensing and acting is another issue relevant 
to our work. There are several approaches using the situation calculus to deal with this 
problem. For example, formalizing a planning problem, one can decide the relevance 
of unexpected changes for an actual plan |9|. The authors of [ 10| employ situation cal- 
culus as well as decision-theoretic planning to control an autonomous robot. Whenever 
passive sensing differs from expectations, re-planning is initiated. In JTTJ Thielscher 
et al. presented a formalization using the fluent calculus to derive explanations for un- 
expected situations. Their approach employs active sensing to minimize the impact of 
such events. Weld et al. IT2l extended GraphPlan to handle sensing actions and uncer- 
tainty, but they do not maintain the consistency of a system's belief. 



3 History-Based Diagnosis and Belief Management for IndiGolog 

In this section we present a belief management system that ensures the consistency of 
a robot's belief with reality (as it is perceivable), preceded by necessary preliminaries. 
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3.1 Situation Calculus, IndiGolog and History-Based Diagnosis 

IndiGolog (Incremental Deterministic Golog) |[T3l is a logic-based programming and 
planning language used for agents and robots. It lets the user choose whether com- 
putational resources should favor planning or imperative programs, and is based on 
Reiter's variant of the situation calculus lfT4l . With the situation calculus, all activities 
and changes in the world are encoded as actions. A situation is therefore completely 
determined by some initial situation and an action sequence (that is also called history). 
This entails that the terms situation and history are interchangeable. Properties are de- 
scribed by situation-dependent predicates and functions, named fluents. The user speci- 
fies dependencies between a fluent and an action by successor state axioms (SSAs). The 
so-called basic action theory (BAT) ifPfll models the environment and the robot's capa- 
bilities. IndiGolog programs are interpreted by a transition semantics in a step-by-step 
fashion. The semantics of program statements are defined by Trans and Final predi- 
cates. Final{a, s). 

Program interpretation in IndiGolog is based on a five step main cycle 1T31 : 

1 . All exogenous events reported in the last cycle are integrated into the history. 

2. The currently considered history fragment is moved forward if necessary. Taking 
only a fragment into consideration keeps the history at a processable length. 

3. The program a is checked if it may terminate in the current situation s using Final. 

4. The program a evolves a single step using the predicate Trans, a primitive action is 
executed if applicable and the history changes accordingly. 

5. Sensing results obtained in the current cycle are integrated into the history. 

This cycle is repeated until either the program terminates legally or there is no further 
executable legal transition. 

In 1 3 1, Iwan describes a diagnosis approach, named history-based diagnosis (HBD), 
for robots in dynamic environments. HBD generates action sequences (histories) that 
are consistent with some observation <P that contradcits the original history. Based on 
the situation calculus, alternative histories are generated by two operations: (1) variation 
of individual actions in the original sequence (action and sensing faults), and (2) inser- 
tion of exogenous actions. In order to isolate the most attractive history out of a possibly 
large set of hypotheses, probabilities and the amount of variation from the original one 
are used to rank the alternatives. 

3.2 A Belief Management System for Dynamic Environments 

We designed and implemented our belief management system around history-based 
diagnosis and IndiGolog. For our purpose, we had to adapt the standard features and the 
interpretation cycle of IndiGolog by adding diagnostic reasoning and related decision 
procedures as well as features to handle multiple hypotheses. 

Our concept for handling multiple hypotheses mainly bases on an adapted IndiGolog 
temporal projector that is able to maintain a pool of hypotheses S (of configurable size) 
as well as a reference to the current favorite hypothesis Sf £ S. While the current steps 
in the IndiGolog cycle were only adapted for the changes to the temporal projector, 
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we added another step la (executed after step 1 of the main cycle) responsible for the 
control of our belief maintenance system. 
The main stages of our new step la are 

1. update all s, 6 S : Sj ^ s/ 

2. check consistency of s/ 

3. perform diagnosis if necessary 

4. decide on a new s/ if necessary and proceed with stage 2 of the main cycle 

The first stage is necessary as steps one and five of the standard cycle deal only with Sf 
and we have to adopt related changes for the alternative hypotheses in S as well. 

In Stage 2 we verify Sf's consistency with reality. For this purpose background 
knowledge (common sense, domain knowledge) is used to derive new facts and con- 
flicts from perception results while executing a plan it. Although results from percep- 
tion can be handled by SSAs the same way as exogenous or endogenous actions, they 
do not change the environment. Thus every fluent update resulting from a sensing ac- 
tion has to be checked whether it masks an inconsistency. This can be done by assuring 
that a fluent is not restricted in such a fashion that would contradict the set of values 
proceeding the sensing action. 

Stage 3 triggers a diagnosis process in the case that Sf is found to be inconsistent. 
In this case, we derive diagnoses satisfying the definition in Section [3~T1 As already 
pointed out, a history is made up of endogenous, exogenous and sensing actions. En- 
dogenous and sensing actions are performed actively, so it is a good guess for it or some 
variation to be part of the correct hypothesis, whereas exogenous actions can occur any- 
time and thus have to be considered with specific conditions. 

We adapted history-based diagnosis according to Iwan [ 3 1 with these ideas in mind 
and created a predicate Varia(OrigAct, Var, Cond, PV) that is used to generate a valid 
variation Var of some action OrigAct with a preference value PV if condition Cond is 
met by situation s. The preference value is one of {1, 2}, where the higher the value 
the less likely this variation occurs. We use a value of 2 for incomplete initial situations 
(which we discuss later) and a value of 1 otherwise. The preference value of a derived 
sequence is defined as the sum of the individual PVs. 

Derived action sequences have to be feasible, that is, it has to be possible to follow 
them step-by-step from a given initial state. As this implies that the initial assignment 
for all relevant fluents has to be known, our system has to be able to deal with incom- 
plete initial belief. We cope with this issue by calculating also possibly executable hy- 
potheses under the assumption of an incomplete Sq and assign them higher preference 
values. As in general the number of derivable situations is very large, we apply pruning 
and abandon (partly) generated situation whenever they reach a predefined threshold. 

In stage 4, we decide on a new Sf in case the actual one is found to be inconsistent. 
The preference value derived in stage 3 for each s, G S is used for a ranking, where 
in a straight forward approach we choose that hypothesis with the highest rank. Unless 
there is no valid candidate for Sf, execution is continued with stage 2. 

With our basic concept we provide an effective belief management system that is 
able to handle multiple hypotheses and several type of faults, and that can be easily 
integrated into IndiGolog. First experiments are discussed in the next section. 
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4 Experimental Analysis 



We simulated and analyzed the performance of our belief management system for the 
scenario of an office delivery robot, a promising application area for robotics. The robot 
has to move objects between 59 rooms that are connected via 12 hallway segments. We 
simulate a robot that is able to pick up an object, move between adjacent rooms, and 
put the object currently carried down again. The robot is equipped with a vision sensor 
registering objects within the same room, and a pressure sensor indicating whether it 
currently carries an object. In order to resemble real robot systems more closely, con- 
tinuous sensing was implemented as a new feature in IndiGolog. 

The SSAs and action pre-conditions were implemented in a straightforward fashion. 
Our background model states basic rules, for example that the robot has to currently 
carry some object if the pressure sensor signals it. In order to prevent influences from 
planning, we use a simple IndiGolog program, that chooses the next action to execute 
out of a given sequence. Typical execution traces consist of about 40 steps. 

We defined three fault scenarios: In Fl the robot might fail to pick up an object with a 
probability of 40% and pick up the wrong object with 20%. In F2 the robot additionally 
fails to release an object with a probability of 30%. In F3 we add a probability of 5% that 
a sensing action fails. These faults resemble issues that frequently occur in reality. We 
define three sensing rates: In SI there is sensor data after every, in S2 after every second, 
and in S3 after every third action. For a single mission, the robot has to deliver three 
particular objects to three particular offices. A mission is accomplished if at the end all 
objects are at the desired locations. It is failed if the interpreter aborts the program, the 
execution exceeds a 2 minute time-out, or some object ends up misplaced. 

A simple base line agent and another one equipped with our belief management sys- 
tem have been tested. They had to solve 50 different missions 10 times each with vary- 
ing seeds for any combination of fault and sensing rates. The percentage of successful 
missions serves as performance measure. 

The results of our experiments are reported in Figure Q] The robot using our belief 
management system significantly outperforms the robot without this extension. From 
the figure, it is also evident that while with rising difficulty (more fault types) the per- 
formance of both robots declines, belief management helps to keep it at a decent level. 
A declining sensing rate has a negative impact on performance too, based on the fact 
that the robot has less information to detect and repair inconsistent situations. 
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Fig. 1. Successfully completed missions with (BMS) and without belief management (Base). 
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5 Conclusions and Future Work 

In this paper we presented a belief management system that seamlessly integrates into 
the agent programming language IndiGolog. Our basic concept was to integrate history- 
based diagnosis into the interpreter in order to allow it to repair inconsistent situations 
and maintain several situation hypotheses. First experiments with an office delivery 
robot scenario show that a robot equipped with our system is able to handle significantly 
more erroneous situations successfully than a simple robot. In contrast to classical diag- 
nosis approaches, the situation calculus enables us to deal also with faults that occurred 
in the past, rather than just faults in the current situation. 

Future work will include formally proving the correctness of our approach, as well 
as a setup for our real robots. Moreover, we will investigate the run-time properties of 
our approach that are important for a broad applicability of our idea. Finally, we will try 
to move the focus from the described passive behavior to a more active management of 
diagnoses such as hypothesis discrimination via actively performed actions. 
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Abstract. Despite significant progress in the development of virtual soccer 
games, the affective behavior of virtual soccer players, both in software and 
hardware applications, is still limited. To improve this situation, this paper 
presents a generic model for decision making of virtual agents in relation to 
emotions and trust. The model takes the BDI framework as point of departure, 
and extends this with mechanisms to represent the dynamics of emotions and 
trust. After testing the model by means of simulation experiments, it has been 
incorporated into the virtual agents within the RoboCup 2D soccer 
environment. A preliminary evaluation pointed out that the model successfully 
enhances believability of virtual soccer players. 

Keywords: soccer simulation, cognitive modeling, affective computing, trust. 

1 Introduction 

Association football, commonly known as soccer, has been the world's most popular 
sport for many decades. Thanks to the rapid developments in the video game industry 
as well as scientific areas such as Computer Science and Artificial Intelligence, 
nowadays, soccer games are not only played in the real world, but also in the virtual 
world. Overall, the purpose of virtual soccer games (or soccer simulations) is three- 
fold. First and foremost, virtual soccer games are played for entertainment purposes; 
in these cases, one or multiple human users control the soccer players in a virtual 
environment, with the goal to defeat each other or a computer opponent [12], Second, 
soccer simulations are increasingly being used for the purpose of analysis, for 
instance by managers and coaches of real soccer teams; here, the idea is that the 
virtual environment provides an easy framework to test various strategies that cannot 
easily be tested in the real world. For example, in case a coach intends to investigate 
what might happen if his team uses a particular tactical strategy in a particular game, 
he could run a simulation of this game, where each of the simulated players behaves 
according to this strategy. Although this area is yet to be explored in much more 
detail, some anecdotical cases have been reported in which soccer simulations were 
able to make accurate predictions (e.g., [15, 17]). Third, virtual soccer games have 
proved to be an excellent test bed for the application of a variety of AI techniques, 
including (real-time) multi-agent planning, multi-agent communication, behavior 
modeling, and learning. These challenges have been the main motivation for the 
foundation of the RoboCup competition in 1997 [10]. 
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As a result of these developments, realism of virtual soccer games has increased 
very rapidly over the past years. Not only has the physical appearance of players and 
environment become more and more appealing, also the technical movements as well 
as the tactical decisions of the players have become much closer to reality. In contrast, 
one particular aspect that has stayed a bit behind is the players' mental behavior, and 
in particular their affective behavior. For instance, in state-of-the-art video games, the 
only moments in which players' emotions are really apparent are right after a goal has 
been scored. In such cases, players usually show some built-in emotional expressions 
(e.g., cheering, or looking angry). However, complex emotional states that change 
dynamically during the game and influence the players' behavior are usually very 
limited. This is in conflict with the behavior of human soccer players, which has 
shown extremely sensitive to the dynamics of their affective states [7]. Consequently, 
existing soccer simulations are not as realistic (and appealing) as they could be. 

To deal with this problem, the current paper presents a generic model for decision 
making in relation to emotions and trust, and applies this to the domain of virtual 
soccer. As also put forward in [16], endowing virtual soccer players with emotions 
will enhance their believability, thereby making the game more engaging for 
spectators. Both emotions and trust are mental states which have been found to have a 
serious impact on people's decisions, in general as well as in sports context [7, 9]. 
The presented model takes the BDI framework [13] as point of departure, and extends 
this with mechanisms to represent the dynamics of emotions and trust. 

The remainder of this paper is structured as follows. The generic simulation model 
is presented at a conceptual level in Section 2. In Section 3, this model is illustrated 
for the context of soccer games, by showing a number of simulations generated in the 
LEADSTO simulation environment. Next, Section 4 describes how the model was 
incorporated into the virtual agents within the RoboCup 2D soccer environment, and 
presents some preliminary results. Section 5 concludes the paper with a discussion. 



2 Conceptual Model 

In this section, the model for decision making with emotions and trust will be 
described at an intuitive, conceptual level, using the agent-based modeling language 
LEADSTO [2]. This language allows the modeler to integrate both logical 
(qualitative) and numerical (quantitative) aspects. In LEADSTO, direct temporal 
dependencies between two state properties in successive states are modeled by 
executable dynamic properties. The format is defined as follows: let a and P be state 
properties of the form 'conjunction of ground atoms or negations of ground atoms'. In 
LEADSTO the notation a — » e f h |J means: 

If state property aholdsfor a certain time interval with duration g, 

then after some delay (between e andf), state property ft will hold for a certain time interval of length h. 

Here atomic state properties can have a qualitative, logical format, such as an 
expression desire(d), expressing that desire d occurs, or a quantitative, numerical 
format such as an expression has_value(x, v) which expresses that variable x has value 
v. For more details, see [2]. 
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Below, the LEADSTO model for decision making with emotions and trust is 
introduced (at a conceptual level) in three steps. In Section 2.1, the basic Belief - 
Desire-Intention (BD1) model is briefly introduced. Next, in Section 2.2 and 2.3, this 
model is extended with mechanisms to represent the dynamics of emotions and trust, 
respectively. For the complete LEADSTO specification, see the M.Sc. Thesis in [6]. 

2.1 BDI-Model 

The BDI-model is a standard paradigm within the Agent Technology domain, which 
describes how agents decide to perform actions based on their beliefs, desires and 
intentions (e.g., [13]). This model forms the foundation of the model presented in this 
paper. See the middle part of the global overview in Figure 1 . Here the box indicates 
the borders of the agent, the ovals denote components of the agent model representing 
different mental states, and the arrows indicate influences between mental states. 




Fig. 1. Global overview of the BDI-model with Emotions and Trust. 



For instance, an action A is executed when the agent has the intention to perform 
this action and it has the belief that certain circumstances in the world are fulfilled 
such that there is an opportunity to perform the action. Beliefs are created on the basis 
of observations. Moreover, the intention to do a specific type of action is created if 
there is some desire to reach state S, and the agent believes that performing this action 
will fulfill this desire. Such relations within the general BDI-model can be specified 
in formal LEADSTO format as follows (where the timing parameters e, f, g, h, as well 
as a parameter Ag indicating which agent has the mental states, have been omitted for 
simplicity): 

VS:STATE VA:ACTION 

desire(S) a belief(satisfies(A, S)) — » intention(A) 

VA:ACTION 

intention(A) a belief(opportunity_for(A)) — » performed(A) 

Note that the beliefs used here depend on observations (see Figure 1), or on common 
knowledge. In the remainder of this paper, desires and intentions are parameterized 
with a real number I (between and 1) to represent the strength of the mental state. 
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2.2 Emotions 



In order to represent emotional states, a main assumption made is that the intensity of 
different emotions can be represented via a number between and 1, as often done 
within emotion models within Artificial Intelligence (e.g., [5]). In the current paper, 
only one type of emotion is modeled, namely happiness (with = very sad and 1 = 
very happy), but the model can also be used to represent emotions like anger and fear. 
As shown in the upper part of Figure 1, the component to represent emotions 
interacts with the basic BDI-model in various manners. First, for generation of 
emotions, it is assumed that they are the result of the evaluation of particular events 
against the individual's own goals (or desires), as described by appraisal theory [5], 
This process is formalized, among others, by the following LEADSTO rules (stating 
that reaching a goal state S leads to happiness with an intensity that is proportional to 
the strength of the desire, and inversely for not reaching a goal state): 

VS:STATE VLREAL 

desire(S, I) a belief(S) —» happiness(S, I) 

desire(S, I) a belief(not(S)) —» happiness(S, 1-1) 

For example, in case an agent desires to score a goal, but misses, then the agent 
becomes sad about this. In addition to these event-specific emotional states, the 
agents' long term mood is modeled. Moods are usually distinguished from emotions 
in that they last longer and are less triggered by particular stimuli. Thus, in the model 
a global mood is assumed with a value between and 1 (representing its positive 
valence), which depends on the values of all specific emotions in the following way: 

VS1 SmSTATE VI1 In, J:REAL 

happiness(S1 , 11) a ... a happiness(Sn, In) a mood(J) — » 
mood((3 * J + (1-p) * (w1*M + ... + wn'ln)) 

Thus, the new mood state is calculated based on the valence J of the old mood state, 
combined with the weighted sum of the emotional values Ik for the different aspects 
Sk in the domain of application. Here, the wk's (which sum up to 1) are weight factors 
representing the relative importance of the different aspects, and the p (between and 
1) is a persistence factor representing the rate at which mood changes over time. 

Finally, various rules have been formulated to represent the impact of emotions 
and mood on beliefs, desires, and intentions. These rules are mostly domain-specific; 
for instance, a positive mood increases the desire to cooperate with teammates, and a 
negative mood increases the desire to behave aggressively. These rules are described 
in detail in [6] . 

2.3 Trust 

According to [4], trust is a (dynamic) mental state representing some kind of 
expectation that an agent may have with respect to the behavior of another agent or 
entity. The authors propose that a belief in competence of the other entity is an 
important contributor to the development of trust (in addition to a second aspect, the 
belief in willingness; however this aspect is currently ignored). This mechanism is 
formalized in the current paper by assuming that trust in another agent is based on a 
weighted sum of the beliefs in the agent's capabilities (using a similar formula as 
above for mood update): 
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VA:AGENT VX1 Xn:ACTION VI1 In, J:REAL 

belief(has_capability(A, X1, 11)) a ... a belief(has_capability(A, Xn, In)) a trust(A, J) — » 
trust(A, p*J + (1-p)*(w1*l1 + w2*l2 + ... + wn*ln)) 

Thus, the new trust state in agent A is calculated based on the level J of the old trust 
state, combined with the weighted sum of the beliefs in capabilities Ik for the different 
actions Xk in the domain of application. For example, a soccer player trusts his 
teammate more if he believes that he is good at attacking as well as defending. Again, 
the wk's are weight factors, and the p is a persistence factor. 

A next step is to define how beliefs in capabilities are determined. For this, the 
mechanism put forward in [8] is reused, which states that a new trust state in some 
entity is partly based on the old trust state, and partly on an experience. This is 
modeled via the following LEADSTO rule (where the experiences are observed 
actions performed by teammates): 

VA:AGENT VX:ACTION VLREAL 

belief(has_capability(A, X, I)) a observed(performed(A, X, succeeded)) — » 

belief(has_capability(A, X, 1 - y + Y*l)) 
belief(has_capability(A, X, I)) a observed(performed(A, X, failed)) — » 

belief(has_capability(A, X, y*l)) 

For instance, in case agent X believes that agent Y's capability with respect to 
tackling is 0.6, and Y performs a successful tackle, then this belief is strengthened 
(where y is an update speed factor). Note that the mechanism to update trust is also 
applied to the self, to model some kind of self-confidence. 

Finally, as with emotions, trust states also have an impact on the other states in the 
BDI-model. For example, a high trust in a teammate increases the strength of the 
intention to pass the ball to this player. Again, these mechanisms are represented 
using (mostly domain-specific) rules, which can be found in [6]. 

3 Simulation Results 

To test the basic mechanisms of the model, it has been used to generate a number of 
simulation runs within the LEADSTO simulation environment. To this end, various 
scenarios have been established in the context of a (simplified) soccer game. The 
game has been simplified in the sense that we did not simulate a complete soccer 
match (as is the case in the RoboCup environment), including computer-generated 
teammates and opponents, and environmental processes (e.g., movements of the ball). 
Instead, the tests focused on the behavior of one particular agent (player X). To test 
this behavior, the scenarios consisted of a series of predefined events (e.g., 'teammate 
Y passes the ball to teammate Z', 'teammate Z shoots on target'), which were 
provided as input to player X. Based on these inputs, all actions and emotions derived 
by the agent were observed and evaluated, and in case of inappropriate behavior, the 
model was improved (manually). Since the main goal of the simulations was to test 
the model for decision making in relation to emotions and trust, this simplified setup 
was considered a necessary first step. As a next step (see Section 4), the model was 
tested in a more complete setting (the RoboCup environment). 

To represent the domain-specific aspects of virtual soccer, the different logical 
sorts introduced in Section 2 were filled with a number of elements. For example, 
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some instances of the sort STATE were {win„game, ballJn_possession, ball_nearby_goal, 
score_other_goal}, some instances of the sort BELIEF were {is_at_location(AGENT, REAL), 
has_capability(AGENT, ACTION, REAL), satisfies(ACTION, STATE), opportunityJor(ACTION) } , 
and some instances of the sort ACTION were {shoot_at_goal, pass„fromJo(AGENT, 
AGENT), dribble, runjree}. 

To illustrate the behavior of the model, Figures 2 and 3 show some fragments of an 
example LEADSTO simulation trace, addressing a simple scenario in which a number 
of subsequent (positive and negative) events occur. In both figures, time is on the 
horizontal axis. Figure 2 shows a number of actions that are performed during the 
scenario by player Z, a teammate of player X. A dark box on top of a line indicates 
that an action is being performed at that time point. Figure 3 shows the emotional 
state (in this case the amount of happiness, on a [0,1] scale) that player X has with 
respect to player Z. 
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Fig. 2. Example simulation results - Actions performed by player Z. 
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Fig. 3. Example simulation results - Emotional state of player X with respect to player Z. 



As can be seen from the figures, player X's emotional state regarding player Z 
changes as a consequence of player Z's actions. Note that it takes about 3 time points 
in the simulations to change an emotional state, which explains the short delays. For 
instance, at time point 7-9, player Z shoots at the goal. Apparently, this attempt is 
appreciated by player X (i.e., it contributes to fulfillment of this agent's desires), since 
at time point 10-12, the agent's happiness regarding agent Z increases from 0.5 to 
0.55. Similarly, player Z's attempts to pass the ball forward (time point 19-25) also 
lead to an increase of agent X's happiness. Note that not all of player Z's actions lead 
to an increase of happiness. There are two potential causes for this: either the action 
fails (not shown in the figure), or the action does not contribute to player X's desires. 
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In addition to the scenario shown above, various other scenarios have been 
simulated in a systematic manner (varying from scenarios in which only positive or 
negative events occur to scenarios in which both positive and negative events occur). 
Due to space limitations, the details of these simulations are not shown here. To 
improve the model's accuracy, each simulation run has been evaluated in the 
following manner. Based on the literature in decision making, emotion, and trust (see, 
e.g., [4, 5] and references in those papers), a number of requirements for the behavior 
of the agents have been formulated; some examples are the following: 

• 'events that contribute to an agent's goals lead to an increase of happiness' 

• 'agents that perform more successful actions are trusted more' 

• 'happy agents perform different actions than sad agents' 

These requirements have been tested against all simulated traces. In case a 
requirement was not fulfilled, small modifications in the model were made, either by 
correcting bugs in the specification or (more often) by adapting the values of the 
parameters involved. After a number of iterations, the model was considered ready to 
be implemented in a more realistic setting. 

4 RoboCup Application 

After the initial tests mentioned above pointed out that the model showed acceptable 
behavior, it has been implemented within the RoboCup 2D virtual soccer 
environment. RoboCup is an international competition founded in 1997 with the aim 
to develop autonomous (hardware and software) agents for various domains, with the 
intention to promote research and education in Artificial Intelligence [10]. RoboCup 
has four competition domains (Soccer, Rescue, Home, and Junior), each with a 
number of leagues. For the current project, the RoboCup Soccer 2D Simulation 
League was used, since this league provides good possibilities for rapid prototyping 
for games with many inter-player interactions. 

In order to implement the model for decision making with emotions and trust in an 
efficient manner, part of the existing code of the 2005 Brainstormers team [14] was 
reused. This code contains a number of explicit classes for different actions (such as 
'shoot_at_goal', 'pass_from_to', 'dribble', 'tackle', and 'run_free'). To create optimal 
strategies at the level of the team, the Brainstormers used Machine Learning 
techniques to learn which actions were most appropriate in which situations. Instead, 
for the current paper, a different approach was taken: for each individual agent, the 
original action classes were directly connected to the output (the generated actions) of 
the implemented decision making model. Thus, instead of having a team of agents 
that act according to a learned collective strategy, each agent in our implementation 
acts according to its personal role, (partly emotion-driven) desires and intentions 1 . 

This approach turned out to be successful in generating (intuitively) realistic soccer 
matches, in which the behavior of the players is influenced by their emotions and trust 
states. A screenshot of the application is shown in Figure 4. The top of the figure 



Recall that the aim of the current paper is not to develop optimal strategies, but rather to 
enhance believability of the virtual players. 
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shows the soccer field, including two teams of 1 1 agents (the colored circles) and the 
ball (the white circle), which is held by the goalkeeper on the right. On top of each 
agent, a small number is displayed, which indicates the player's personal number. 
Below, some information is displayed in plain text, to give some insight in the mental 
states of the agents. Obviously, there is no space to display all aspects of the agents' 
mental states (emotions, moods, trust states, beliefs, desires, and intentions). 
Therefore the user can select which information is shown. In the example shown in 
Figure 4, five columns of information are shown, indicating, respectively, the current 
time, and each agent's number, mood, emotion regarding itself, and current action. 
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Fig. 4. Screenshot of the RoboCup application. 



An example animation illustrating the behavior of the application can be found at 
http://www.youtube.com/watch?v=I-ZamC-louo. This movie shows a scenario where 
player 6 passes the ball to player 10, who then dribbles with it towards the goal, and 
scores. As a result of these successful actions, various aspects of player 10's mental 
state change (see the textual information below the playing field): its emotion with 
respect to itself increases (from 0.366 to 0.686), as well as its overall mood (from 
0.398 to 0.557), its trust in itself (from 0.522 to 0.552), and its belief about its own 
capability to shoot at the goal (from 0.509 to 0.602). As a result, in the rest of the 
scenario the player will behave more confidently, and will try to score more often. 

The application has been tested extensively, using a large amount of different 
parameter settings (e.g. for initial trust, emotion and mood states, trust and emotion 
flexibility, and action preferences, see also [6]). In all tests, the behavior of all players 
was analyzed in a step-wise manner, and compared with expectations of the modelers 
(based on common sense). Various interesting types of behavior were observed, 
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which were found to be consistent with behavior as observed in real soccer games. 
For instance, players that had scored a number of goals were trusted more by their 
teammates, and received more passes from them. Also, players with negative 
emotions committed more fouls than other agents. 

To further evaluate the application, 10 participants (experienced video gamers 
between 25 and 32 years old) were asked to judge the behavior of the virtual players, 
by answering questions like "do you think the players behave in a realistic manner?", 
"do you think the players show appropriate emotions?". The initial results of this 
evaluation were promising: the participants very much appreciated the players' 
abilities to show emotions and trust. Overall, they had the idea that the presented 
model made the agents more believable, and that it enhanced the experienced fun 
when watching the soccer games. Nevertheless, in a later stage, a more elaborated 
evaluation experiment will be performed, in cooperation with colleagues from Social 
Sciences. In this experiment, we plan to compare virtual players that use different 
variants of the presented model with players that do not have emotions and trust. 

5 Discussion 

To enhance believability of virtual soccer players' affective behavior, this paper 
presented a generic model for decision making of virtual agents in relation to 
emotions and trust. The backbone of the presented model is the BDI framework, 
which was extended with mechanisms to represent the dynamics of emotions and 
trust. After testing the model by means of simulation experiments, it has been 
incorporated into the virtual agents within the RoboCup 2D soccer environment. 
Although preliminary, an initial evaluation pointed out that the model has the 
potential to enhance believability of virtual soccer players. 

Related work in the area of believable virtual soccer players is scarce. Since the 
initiation of the RoboCup competition in 1997, a number of participants focused on 
the cognitive aspects of the players (e.g., [3, 11]), but emotional aspects were mostly 
ignored. A welcome exception is presented by Willis [16], who proposes an 
architecture to endow the robots in the physical league with emotional intelligence. 
Unlike the model put forward in the current paper, this architecture has not been 
implemented yet, nor does it address the concept of trust. Finally, some papers (e.g., 
[1]) address implementation of emotions within the spectators and commentators of 
virtual soccer games; however, to the best of our knowledge, these approaches have 
never been applied to the player agents. 

As mentioned in the introduction, endowing virtual soccer players with more 
human-like behavior can be useful for several reasons (see [16] for an extensive 
overview): for instance, the games become 1) more fun to watch and 2) more faithful 
to reality, which makes it possible to use them as an analytical tool for coaches. A 
third reason put forward in [16] is that soccer teams with emotional intelligence may 
have a competitive advantage over other teams. Although this was not the main focus 
of the current paper, future research may investigate how well our agents perform 
against similar agents without emotions and trust. 

Other future work will include a further elaboration of the model, among others, by 
focusing on other emotions such as anger and fear, and the interaction between 
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multiple emotions. Also, possibilities will be explored to visualize players' emotional 
states in such a way that they are easily observed by spectators. For instance, a 
straightforward way to implement this would be using different colors to represent 
different emotions. It is expected that these extensions will enhance spectators' 
engagement in virtual soccer games even further. 
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